Rotating object detection

Abstract

The at least one processor determines a bounding box in the image using a neural network. The at least one processor extracts an initial bounding box of an object in the image. The at least one processor selects a cluster center for the bounding box and defines a cluster box around the cluster center. The at least one processor rotates the initial bounding box and contents of the initial bounding box according to a rotation angle determined based on the cluster box. The at least one processor detects the object in the rotated initial bounding box using a trained machine learning model.

Description

Background Technology

[0001] This application relates generally to computers and computer applications, and more specifically to machine learning and object detection (such as rotational object detection). Summary of the Invention

[0002] This disclosure is provided to aid in understanding computer systems and methods for rotating object detection and is not intended to limit the scope of the disclosure or the invention. It should be understood that in some cases, various aspects and features of this disclosure may be used advantageously alone, or in others in combination with other aspects and features. Therefore, variations and modifications can be made to the computer system and its operation to achieve different effects.

[0003] The computer-implemented method in some embodiments includes using a neural network to determine bounding boxes in an image. The computer-implemented method also includes extracting initial bounding boxes of objects in the image. The computer-implemented method further includes selecting cluster centers of the bounding boxes and defining cluster boxes around the cluster centers. The computer-implemented method also includes rotating the initial bounding boxes and their contents according to a rotation angle determined based on the cluster boxes. The computer-implemented method further includes using a trained machine learning model to detect objects within the rotated initial bounding boxes.

[0004] This document also discloses computer systems and computer program products configured to implement or cause the above methods.

[0005] Further features, as well as the structure and operation of various embodiments, are described in detail below with reference to the accompanying drawings. In the drawings, similar reference numerals denote the same or functionally similar elements. Attached Figure Description

[0006] Figure 1 An example of a computational environment that can implement rotating object detection in some embodiments is shown.

[0007] Figure 2A Example images with bounding boxes are shown in some embodiments.

[0008] Figure 2B Example images with bounding boxes including rotated objects are shown in some embodiments.

[0009] Figure 3 This is a diagram illustrating a method for detecting a rotating object in some embodiments.

[0010] Figure 4A The bounding box of an example image is shown in some embodiments.

[0011] Figure 4B The bounding box is shown as being further filtered in some embodiments.

[0012] Figure 4C Clustering points and outliers of bounding boxes are shown in some embodiments.

[0013] Figure 5 The diagram illustrates the use of a graph cut algorithm to extract the content region of a bounding box in some embodiments.

[0014] Figure 6 Examples of the original detection bounding boxes, cluster boxes, and region of interest (ROI) grab boxes are shown in some embodiments.

[0015] Figure 7 Examples of components of a computer system that can perform rotating object detection in some embodiments are shown. Detailed Implementation

[0016] In one aspect, a computer-implemented method is provided for determining bounding boxes in an image using a neural network, extracting initial bounding boxes of objects in the image, selecting cluster centers of the bounding boxes and defining cluster boxes around the cluster centers, rotating the initial bounding boxes and their contents according to a rotation angle determined based on the cluster boxes, and using a trained machine learning model to detect objects within the rotated initial bounding boxes.

[0017] This method allows for the detection of rotating objects in images without retraining a machine learning model (such as a neural network) with new training data to include different rotational views of the object. This, for example, reduces the amount of training data needed to train the machine learning model. Reducing the amount of data that needs to be processed (e.g., the size of the training data) frees up computational resources to perform other processing tasks, and can therefore improve computational speed. Furthermore, once trained, the machine learning model can be used to detect rotating objects without retraining it with additional training data, further mitigating the use of computational resources.

[0018] One or more of the following features may be separate from or optional. For example, in some embodiments, the computer-implemented method further includes extracting a region of interest (ROI) grasping box by iteratively performing a graph cut algorithm starting with clustered boxes as initial regions and obtaining segmented regions, wherein the segmented regions are used as initial regions in the next iteration, and wherein the iteration continues until a criterion is met; and adjusting the size of the initial bounding box based on the ROI grasping box, wherein a trained machine learning model is used to detect objects occurring within the sized initial bounding box. This allows for the identification of bounding boxes that are more focused on potentially rotated objects.

[0019] In some embodiments, the computer-implemented method further includes training the neural network by using the intersection-over-union value as a mask for the true positive determination of the loss function for the neural network. This allows, for example, the detection of additional bounding boxes, thereby enabling the discovery of those bounding boxes containing objects in the image with higher confidence.

[0020] In some embodiments, a non-maximum suppression process is performed to select initial bounding boxes from the bounding boxes. This identifies the bounding boxes with the highest confidence for containing objects in the image.

[0021] In some embodiments, selecting cluster centers for bounding boxes includes determining representative pixels within each bounding box, clustering the representative pixels of the bounding boxes, and identifying clusters of representative pixels with the highest density regions. This defines and identifies pixel groups to aid in automated preprocessing for computer vision and image recognition.

[0022] In some embodiments, a criterion is met if the size difference between the segmented region and the initial region is less than a predetermined threshold. Thus, the method controls when to stop iteration, for example, based on the predetermined threshold, and the alternation between estimation and parameter learning helps to perform energy minimization rather than performing a single learning iteration.

[0023] In some embodiments, the rotation angle of the initial bounding box is determined by finding the central axis of the initial bounding box and repeatedly rotating the central axis by a certain angle until the pixels in the cluster box are divided into two equal parts by the central axis, wherein the angle that divides the pixels in the cluster box into two equal parts by the central axis is used as the target rotation angle. Thus, for example, the rotation angle can be identified based on the center point of the candidate bounding box of an object that appears to be rotating in the image.

[0024] In some embodiments, adjusting the size of the initial bounding box includes adjusting the edges of the initial bounding box based on the intersection-union ratio (IU) of the region of interest (ROI) and the initial bounding box. This better aligns the initial bounding box with the content within the ROI, thereby improving bounding box positioning around objects that appear to rotate in the image.

[0025] A system is also disclosed, comprising at least one computer processor and at least one memory device coupled to the at least one computer processor, wherein the at least one computer processor is configured to perform one or more of the methods described above. A computer program product is also disclosed, comprising a computer-readable storage medium having program instructions embodied therein, wherein the program instructions are readable by a device to cause the device to perform one or more of the methods described above.

[0026] Various aspects of this disclosure are described by narrative text, flowcharts, block diagrams of computer systems, and / or block diagrams of machine logic included in embodiments of a computer program product (CPP). Regarding any flowchart, depending on the technology involved, operations may be performed in a different order than that shown in a given flowchart. For example, again according to the technology involved, two operations shown in consecutive flowchart blocks may be performed in reverse order, as a single integrated step, simultaneously, or in a manner that at least partially overlaps in time.

[0027] Computer Program Product Embodiment (“CPP Embodiment” or “CPP”) is a term used in this disclosure to describe any collection of one or more storage media (also referred to as “media”) collectively included in a collection of one or more storage devices, the collection of one or more storage devices collectively including machine-readable code corresponding to instructions and / or data for performing computer operations specified in a given CPP claim. A “storage device” is any tangible device capable of holding and storing instructions used by a computer processor. Without limitation, a computer-readable storage medium can be an electronic storage medium, a magnetic storage medium, an optical storage medium, an electromagnetic storage medium, a semiconductor storage medium, a mechanical storage medium, or any suitable combination of the foregoing. Some known types of storage devices that include these media include: magnetic disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), compact disc read-only memory (CD-ROM), digital versatile disc (DVD), memory sticks, floppy disks, mechanical encoding devices (such as punch cards or pits / platforms formed in the main surface of the disk), or any suitable combination of the foregoing. Computer-readable storage media, as used in this disclosure, should not be construed as storing transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides, optical pulses through fiber optic cables, electrical signals transmitted through wires, and / or other transmission media. As those skilled in the art will understand, data is typically moved at certain incidental points in time during the normal operation of the storage device, such as during access, defragmentation, or garbage collection; however, this does not render the storage device transient, as the data is not transient when it is stored.

[0028] The computing environment 100 includes examples of environments for executing at least some of the computer code involved in performing the inventive methods, such as the rotating object detection code 200. In addition to the rotating object detection code 200, the computing environment 100 includes, for example, a computer 101, a wide area network (WAN) 102, an end-user device (EUD) 103, a remote server 104, a public cloud 105, and a private cloud 106. In this embodiment, the computer 101 includes a processor set 110 (including processing circuitry 120 and a cache 121), a communication structure 111, volatile memory 112, persistent storage 113 (including an operating system 122 and the rotating object detection code 200, as described above), a peripheral device set 114 (including a user interface (UI) device set 123, storage 124, and an Internet of Things (IoT) sensor set 125), and a network module 115. The remote server 104 includes a remote database 130. Public cloud 105 includes gateway 140, cloud orchestration module 141, host physical machine set 142, virtual machine set 143, and container set 144.

[0029] Computer 101 can take the form of a desktop computer, laptop computer, tablet computer, smartphone, smartwatch or other wearable computer, mainframe computer, quantum computer, or any other form of computer or mobile device now known or to be developed in the future capable of running programs, accessing networks, or querying databases such as remote database 130. As is well known in the field of computer technology, and depending on that technology, the execution of computer-implemented methods can be distributed among multiple computers and / or multiple locations. On the other hand, in this presentation of computing environment 100, the detailed discussion focuses on a single computer, specifically computer 101, to keep the presentation as simple as possible. Computer 101 can reside in the cloud, even... Figure 1 It is not shown that it is in the cloud. On the other hand, unless explicitly instructed otherwise, computer 101 is not required to be in the cloud.

[0030] Processor set 110 includes one or more computer processors of any type now known or to be developed in the future. Processing circuitry 120 may be distributed across multiple packages, such as multiple cooperating integrated circuit chips. Processing circuitry 120 may implement multiple processor threads and / or multiple processor cores. Cache 121 is memory located within the processor chip package and is typically used for data or code that should be available for fast access by the threads or cores running on processor set 110. Cache memory is typically organized into multiple levels based on its relative proximity to the processing circuitry. Alternatively, some or all of the cache in the processor set may be located “off-chip.” In some computing environments, processor set 110 may be designed to work with qubits and perform quantum computing.

[0031] Computer-readable program instructions are typically loaded onto computer 101 to cause the processor set 110 of computer 101 to perform a series of operational steps to implement a computer-implemented method, such that the instructions thus executed instantiate the method specified in the flowcharts and / or narrative descriptions of the computer-implemented method included in this document (collectively, the “inventive method”). These computer-readable program instructions are stored in various types of computer-readable storage media, such as cache 121 and other storage media discussed below. The program instructions and associated data are accessed by the processor set 110 to control and direct the execution of the inventive method. In computing environment 100, at least some of the instructions for performing the inventive method may be stored in rotating object detection code 200 in persistent storage 113.

[0032] Communication structure 111 is a signal transmission path that allows various components of computer 101 to communicate with each other. Typically, this structure consists of switches and conductive paths, such as switches and conductive paths forming buses, bridges, physical input / output ports, etc. Other types of signal communication paths can be used, such as fiber optic communication paths and / or wireless communication paths.

[0033] Volatile memory 112 is any type of volatile memory now known or to be developed in the future. Examples include dynamically typed random access memory (RAM) or statically typed RAM. Typically, volatile memory 112 is characterized by random access, but this is not required unless explicitly indicated. In computer 101, volatile memory 112 is located in a single package and is internal to computer 101; however, alternatively or additionally, volatile memory may be distributed across multiple packages and / or located externally relative to computer 101.

[0034] Persistent storage 113 is any form of non-volatile storage for a computer, now known or to be developed in the future. The non-volatility of this storage means that the stored data is retained regardless of whether power is supplied to the computer 101 and / or directly to the persistent storage 113. Persistent storage 113 may be read-only memory (ROM), but typically at least a portion of persistent storage allows for data writing, data deletion, and data rewriting. Some common forms of persistent storage include disks and solid-state storage devices. Operating system 122 may take several forms, such as various known proprietary operating systems or operating systems employing an open-source portable operating system interface type with a kernel. The code included in the rotating object detection code 200 generally includes at least some of the computer code involved in performing the inventive methods described herein.

[0035] Peripheral device set 114 includes a collection of peripheral devices for computer 101. Data communication connections between peripheral devices and other components of computer 101 can be implemented in various ways, such as Bluetooth connectivity, near field communication (NFC) connectivity, connections made by cables (such as Universal Serial Bus (USB) type cables), plug-in connections (e.g., secure digital (SD) cards), connections made via local area communication networks, and even connections made via wide area networks such as the Internet. In various embodiments, UI device set 123 may include components such as displays, speakers, microphones, wearable devices (such as glasses and smartwatches), keyboards, mice, printers, touchpads, game controllers, and haptic devices. Storage 124 is an external storage device, such as an external hard drive, or a pluggable storage device, such as an SD card. Storage 124 can be persistent and / or volatile. In some embodiments, storage 124 may take the form of a quantum computing storage device for storing data in the form of qubits. In embodiments where computer 101 requires a large amount of storage (e.g., where computer 101 locally stores and manages a large database), this storage can be provided by peripheral storage devices designed to store very large amounts of data, such as a storage area network (SAN) shared by multiple geographically distributed computers. The IoT sensor set 125 consists of sensors that can be used in IoT applications. For example, one sensor could be a thermometer, while another could be a motion detector.

[0036] Network module 115 is a collection of computer software, hardware, and firmware that allows computer 101 to communicate with other computers via WAN 102. Network module 115 may include hardware such as a modem or Wi-Fi transceiver, software for packetizing and / or depacketizing data for transmission over the communication network, and / or web browser software for transmitting data over the Internet. In some embodiments, the network control and network forwarding functions of network module 115 are performed on the same physical hardware device. In other embodiments (e.g., embodiments utilizing software-defined networking (SDN), the control and forwarding functions of network module 115 are performed on physically separate devices, such that the control function manages several different network hardware devices. Computer-readable program instructions for performing the methods of the invention can typically be downloaded to computer 101 from an external computer or external storage device via a network adapter card or network interface included in network module 115.

[0037] WAN 102 is any wide area network (e.g., the Internet) capable of transmitting computer data over non-local distances using any technology known now or developed in the future for transmitting computer data. In some embodiments, WAN 102 may be replaced by and / or supplemented by a local area network (LAN) designed to transmit data between devices located in a local area, such as a Wi-Fi network. WANs and / or LANs typically include computer hardware such as copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and edge servers.

[0038] End User Equipment (EUD) 103 is any computer system used and controlled by an end user (e.g., a customer of the enterprise operating computer 101) and can take any of the forms discussed above in conjunction with computer 101. EUD 103 typically receives helpful and useful data from the operation of computer 101. For example, assuming computer 101 is designed to provide recommendations to the end user, these recommendations are typically transmitted from network module 115 of computer 101 to EUD 103 via WAN 102. In this way, EUD 103 can display or otherwise present recommendations to the end user. In some embodiments, EUD 103 can be a client device, such as a thin client, a thick client, a mainframe computer, a desktop computer, etc.

[0039] Remote server 104 is any computer system that provides at least some data and / or functionality to computer 101. Remote server 104 can be controlled and used by the same entity operating computer 101. Remote server 104 represents a machine that collects and stores helpful and useful data used by other computers, such as computer 101. For example, if computer 101 is designed and programmed to provide recommendations based on historical data, that historical data can be provided to computer 101 from a remote database 130 of remote server 104.

[0040] Public cloud 105 is any computer system that can be used by multiple entities, providing on-demand availability of computer system resources and / or other computing capabilities (especially data storage (cloud storage) and computing power) without direct active management by users. Cloud computing typically leverages resource sharing to achieve scalability consistency and economy. Direct and active management of the computing resources of public cloud 105 is performed by the computer hardware and / or software of cloud orchestration module 141. The computing resources provided by public cloud 105 are typically implemented by virtual computing environments running on various computers constituting host physical set 142, which is the entirety of physical computers in and / or available to the public cloud 105. Virtual computing environments (VCEs) typically take the form of virtual machines from virtual machine set 143 and / or containers from container set 144. It should be understood that these VCEs can be stored as images and can be transferred between various physical machine hosts as images or after the VCEs are instantiated. Cloud orchestration module 141 manages the transfer and storage of images, deploys new instantiations of VCEs, and manages active instantiations of VCE deployments. Gateway 140 is a collection of computer software, hardware, and firmware that allow public cloud 105 to communicate via WAN 102.

[0041] Now, we will provide some further explanation of Virtualized Computing Environments (VCEs). A VCE can be stored as an "image." A new active instance of a VCE can be instantiated from this image. Two common types of VCEs are virtual machines and containers. A container is a VCE that uses operating system-level virtualization. This refers to an operating system feature where the kernel allows multiple isolated user-space instances, called containers, to exist. From the perspective of the programs running within them, these isolated user-space instances typically appear as actual computers. Computer programs running on a regular operating system can utilize all the resources of that computer, such as connected devices, files and folders, network shares, CPU power, and quantifiable hardware capabilities. However, programs running within a container can only use the contents of the container and the devices allocated to the container; this is a characteristic known as containerization.

[0042] Private cloud 106 is similar to public cloud 105, except that computing resources are available only to a single enterprise. While private cloud 106 is depicted as communicating with WAN 102, in other embodiments, private cloud may be completely disconnected from the Internet and accessible only via a local / private network. A hybrid cloud is a combination of multiple clouds of different types (e.g., private, community, or public cloud types) typically implemented by different vendors. Each of the multiple clouds remains a separate and discrete entity, but the larger hybrid cloud architecture is bound together by standardization or proprietary technology that enables orchestration, management, and / or data / application portability across the multiple component clouds. In this embodiment, public cloud 105 and private cloud 106 are both part of a larger hybrid cloud.

[0043] In some embodiments, computer 101 also hosts one or more machine learning models, such as object detection machine learning models. In one embodiment, the machine learning models are stored in persistent storage 113 of computer 101. Received images are transmitted within computer 101 to the machine learning models, for example, via communication structure 111, to different memory areas hosting the machine learning models.

[0044] In some embodiments, one or more machine learning models are stored in computer memory located remotely from computer 101, such as in remote server 104 or in end-user device 103. In this embodiment, code 200 works remotely with the machine learning model to train and use it. Training instructions are sent via a transmission that begins at computer 101, passes through WAN 102, and ends at the destination computer hosting the machine learning model. Thus, in some embodiments, code 200 at computer 101 or another instance of software at a central remote server performs the operation of routing training instructions to multiple servers / geographical locations in a distributed system.

[0045] In such an embodiment, a remote machine learning model is configured to send its output back to computer 101, such that inferences and rotational object detection results provided by analyzing rotated object image portions using trained models, neural networks, and / or object rotators are provided and presented to the user. The machine learning model receives copies of new data samples, performs machine learning analysis on the received samples, and transmits the results (e.g., outputs such as classifications of detected objects) back to computer 101 for presentation there, for example, via a display of UI device set 123.

[0046] While traditional machine learning or computer vision object detection methods and annotation tools support both vertical and horizontal object detection and calibration, many objects in real-world scenarios appear at arbitrary angles. For example, objects can be placed in different locations and images can be captured from different rotational perspectives. Systems and methods for object detection are disclosed to detect objects appearing at different rotational angles in images (such as, but not limited to, satellite cloud images, navigation, and other scenes). In some embodiments, such systems and methods do not require additional annotation workload when performing object detection on rotated objects.

[0047] In some embodiments, the system and / or method trains an object detection machine learning model using annotations of vertically or horizontally placed objects, receives a new image, automatically detects objects rotated relative to the vertical and / or horizontal orientation in the received image, and returns the coordinates and angles of the rotated objects, such as the coordinates and angles of the rotated bounding box. The bounding box is then rotated and input into the trained machine learning model. By inputting an image that has been fully or partially rotated to fit within the vertical and / or horizontal orientation into the machine learning model, the trained machine learning model is allowed to successfully perform object detection. A bounding box is a rectangle surrounding an object in an image. A bounding box has associated attributes such as its position or orientation (e.g., xy pixel coordinates in the image), size (e.g., width and length in pixels), and one or more types (e.g., category) of the objects contained within the bounding box. In response to input into a neural network or some type of machine learning model, the machine learning model or neural network generates a confidence score for the categorized objects within the bounding box. The confidence score is the probability or likelihood that an object of that type is contained within the bounding box. The system and / or method of this embodiment disclosed herein is particularly useful and practical when labeling data can become very tedious. Existing horizontal and / or vertical annotations and methods can still be used to label data, and the systems and / or methods in some embodiments disclosed herein use trained machine learning models to detect or identify objects with rotation, which are trained on images labeled with objects having horizontal and / or vertical orientations, and which do not require any labeled training data for images of objects oriented differently from horizontal and / or vertical orientations.

[0048] For example, an object detection machine learning model is trained to recognize images of airplanes, but the annotated training images include airplanes conventionally (e.g., axis-aligned) oriented in the image view, such as whose longitudinal axis is collinear or parallel to (A) a vertical line passing through the image (typically a rectangular or sometimes square image) or (B) a horizontal line passing through the image. If the captured image contains an airplane, but the airplane is rotated relative to the conventional orientation, for example, the airplane's longitudinal axis passes through and intersects with the various vertical lines or horizontal lines of the image, a machine learning model trained with conventional annotations may recognize the presence of an object in the image, but sometimes or often fails to classify that object as an airplane. Using embodiments disclosed herein, automatic rotation can successfully create altered rotated images, making the components shown in the rotation more recognizable to a machine learning model trained with conventionally oriented objects.

[0049] Figure 2A and 2B Example images related to product location, classification, and counting in a store are shown in some embodiments. Image processing captures images of products, classifies objects in the images (e.g., through object detection using machine learning), and counts objects, for example, for inventory purposes or other uses. To accomplish this inventory process, automated tools should efficiently detect and locate grids (e.g., bounding boxes with objects) in the image. To complete the entire process, the first step is to efficiently detect and locate each grid in the scene. Figure 2A Images with bounding boxes from some embodiments are shown. Figure 2B Some embodiments are shown with Figure 2A Images of bounding boxes that are rotated such that they are not axis-aligned, for example, oriented collinear, parallel, or perpendicular to vertical and / or horizontal lines that pass through the image.

[0050] In some embodiments, the system and / or method detects bounding boxes containing rotated objects oriented and / or offset in a manner different from the standard vertical and / or horizontal orientation. For example, an object having a central axis deviating from a vertical line and / or horizontal line passing through the image is oriented and / or offset in a different manner relative to the standard vertical and / or horizontal orientation. Detecting bounding boxes with rotated objects can improve rotated object detection. For example, in some embodiments, the method performs coarse object detection based on general object detection, and then finds the optimal position and size of the bounding box (also called the detection box) by iteratively correcting the angle and size of the bounding box, thereby allowing the altered image to be fed into an object detection machine learning model for accurate detection of objects contained within the bounding box. Typically, in some embodiments, the method includes the following procedures.

[0051] Coarse object detection involves initial localization and bounding of object detection boxes using general or existing object detection algorithms. Bounding box correction and more precise localization continuously fine-tune the object detection boxes identified during coarse object detection to find the optimal position and size of the detection boxes. An evaluation function evaluates the new detection boxes and determines the optimal or improved object detection boxes and rotation angles.

[0052] In some embodiments, various anchor points are set using conventional object detection tools, systems, and / or methods, and original bounding boxes are predicted based on the number of anchor points on the output feature map. In some embodiments, anchor points refer to attributes such as graphic coordinates, object scores (e.g., foreground objects or background), and category scores associated with a specific number of categories. During this process, multiple candidate boxes are generated at each location in the feature map. In some embodiments, the system and / or method defines an object detection loss function with soft labels to identify potential bounding boxes. A “soft label” refers to a more flexible or probabilistic representation of the true situation or true label associated with the bounding box or object category. The system and / or method then estimates the rotation angle of the detected object using redundant candidate boxes and the current content of the candidate boxes. A neural network is trained using a modified loss optimization, where soft labels (probabilistic scores) are used as true positives instead of always using “1” as the true positive score. A neural network trained in this way helps identify more boxes as rotation candidates and contributes to higher object confidence during prediction / inference. A “redundant candidate box” refers to multiple bounding boxes that cover the same or similar regions within an image. "Current content of the candidate bounding box" refers to the information contained within a specific bounding box generated by an object detection algorithm or tool at a stage during the detection process. The system and / or method uses one or more evaluation functions to determine the appropriate angle for rotating the content of the detected bounding box, then rotates the detected bounding box by the determined rotation angle and inputs the rotated image into a machine learning model for machine learning object detection.

[0053] Figure 3 This diagram illustrates methods for detecting rotated objects in some embodiments. For example, in some embodiments, existing object detection methods or tools are improved or modified so that candidate bounding boxes with low confidence values, which are typically filtered out, are retained in the candidate bounding box pool by changing the object loss function (e.g., the loss function of a neural network) with soft labels. Rotation features are extracted based on the candidate bounding boxes. Additionally, rotation features are extracted based on the content of the bounding boxes. The bounding boxes are then rotated and adjusted.

[0054] At 302, the method includes using a neural network to determine bounding boxes of objects in an image, for example, by improving or modifying an existing object detection method. Examples of such existing object detection methods include end-to-end neural networks that predict bounding boxes and probabilities of detected objects. For example, a machine learning model (e.g., a neural network, such as a deep convolutional neural network) is trained using a training dataset containing multiple images of objects to be able to detect these objects in previously unseen images during the inference phase. Such a trained neural network receives images as input during its inference phase and detects objects in the images. This object detection or prediction can be performed in real time. In some embodiments, modifying an existing object detection method includes modifying the calculation of the object loss performed by the neural network in the existing object detection method, thereby improving the loss of the object detection method. For example, existing object detection methods may determine whether candidate bounding boxes include “foreground” or “background” pixels based on an Intersection over Union (IoU) threshold. In short, the Intersection over Union (IoU), also known as the Jaccard index, is a commonly used metric for determining how accurate the segmentation of candidate images is. Typically, in the loss function, false positive (FP) foreground boxes (candidate bounding boxes designated as false positive foregrounds) are suppressed with a mask of -1, and true positive (TP) foreground boxes (candidate bounding boxes designated as true positive foregrounds) are rewarded with a mask of 1. In some embodiments, the systems and / or methods disclosed herein change the mask of true positives (TP) in the loss function to an IoU value and compute the mask of true positives (TP) as a soft label, e.g., one or more probability scores within a range instead of a binary label. This soft label allows the object detection model to be trained to regress toward a higher IoU value between the predicted bounding boxes and the ground truth bounding boxes, rather than simply relying on binary classification (correct or incorrect). By using IoU as a soft label, the model is encouraged not only to learn the presence of predicted objects but also to refine bounding box predictions for more accurate localization by considering the degree of overlap between the predicted and ground truth bounding boxes. This mechanism allows more candidate boxes to obtain higher object confidence during prediction. For example, using Figure 2A The sample images shown can be used to apply an improved object loss function to existing object detection methods. Figure 4A The bounding box determined at 302 using an improved object loss function is shown in some embodiments.

[0055] At position 304, a neural network is used to extract or determine the initial bounding boxes of objects in the image. For example, existing object detection methods or neural networks detect the initial bounding boxes of objects in an image. Such initial bounding boxes are extracted or determined based on selecting the bounding box with the highest confidence value among the candidate bounding boxes associated with the object.

[0056] For example, in some embodiments, a non-maximum suppression (NMS) process is used to select a single initial bounding box associated with an object from candidate bounding boxes associated with that object. In short, the NMS process is a computer vision method that selects a single entity with the highest confidence score from a plurality of overlapping entities (e.g., bounding boxes in object detection). Entities that significantly overlap with the selected bounding box (e.g., greater than a predetermined threshold) are removed. In some embodiments, the NMS process is performed to obtain an initial bounding box for each detected object. For ease of explanation, the initial bounding box of an image is referred to as “R1”. For example, Figure 2A This illustrates multiple initial bounding boxes corresponding to multiple objects in an image, extracted by feeding an image into a neural network (such as a convolutional neural network).

[0057] At 306, the method includes selecting cluster centers of bounding boxes and defining cluster boxes around the cluster centers. In some embodiments, the method extracts boxes as rotation candidates and selects cluster centers for each extracted candidate box. A "box as a rotation candidate" refers to a box that has successfully passed some filtering process used to remove unsuitable bounding boxes. The extracted boxes can be used to estimate the rotation angle of the detected object located within the respective extracted boxes. For example, in some embodiments, extracting boxes as rotation candidates includes obtaining all boxes associated with the object with confidence values ​​greater than a threshold (e.g., 10e-1) prior to a non-maximum suppression (NMS) process, calculating the IoU with the correct candidate box (referred to as "candidate box A" for ease of interpretation), and filtering out candidate bounding boxes with an IoU threshold less than a defined value (e.g., 0.3). The "correct candidate box" or "candidate box A" is a specific candidate box considered as a reference or ground truth associated with the object. The R1 box is an example of such a ground truth or reference box. Noisy boxes with large or long distances are filtered out. Figure 4B It is shown that in some embodiments from Figure 4A The bounding boxes shown are filtered bounding boxes (e.g., boxes that are rotation candidates).

[0058] In some embodiments, the method calculates score-weighted centroids of boxes (for all candidate boxes obtained prior to the NMS process described above and whose confidence values ​​are above a pre-configured threshold, e.g., 10e-1), runs a noisy application-based density-based spatial clustering (DBSCAN) within all centroids, and selects the cluster with the most points. Other points representing noise are removed. A weighted average of the box centroids based on the confidence scores gives the centroids used for box selection. Calculating score-weighted centroids of boxes refers to a method used to calculate representative points within a bounding box, taking into account the confidence score or the score associated with that particular bounding box. By using this score-weighting method, higher-confidence boxes will contribute more to the centroid calculation, which may result in more representative points, considering the confidence level associated with each box while determining the overall centroids of detected objects. In short, DBSCAN is a clustering algorithm for data analysis and pattern recognition. DBSCAN groups data points based on their density—identifying clusters of high-density regions and classifying outliers as noise. Figure 4C Cluster points and outliers of bounding boxes are shown in some embodiments. In some embodiments, the method retains the cluster center 402 (the center of the bounding box) of the bounding box and removes outliers 404. The cluster with the most points is selected because the box tightly wraps the contained objects.

[0059] In some embodiments, the method further includes calculating the bounding rectangle (referred to as "rectangle C" for ease of explanation) based on the selected clusters. For ease of explanation, this is also referred to as a "cluster box" or "R2". The terms "bounding rectangle," "cluster box," or "R2" refer to the bounding box generated based on points clustered together using the DBSCAN algorithm. Calculating the bounding rectangle (e.g., the cluster box or rectangle C) includes performing DBSCAN clustering, selecting clusters, and performing the bounding rectangle calculation (e.g., around the cluster bounding box). The bounding rectangle or cluster box R2 surrounds the object and the selected cluster points. In some cases, the rectangle is generated in a multi-step process, which includes first generating a convex hull around all points of the selected clusters, and then second generating a rectangle to cover the convex hull.

[0060] refer to Figure 3At 308, the method includes extracting rotational features based on bounding box content, for example, by iteratively performing a graph cut algorithm starting with clustered boxes as initial regions in the image and obtaining segmented regions until a criterion is met to extract a region of interest grabbing box. The segmented regions are used as initial regions in the next iteration. For example, in some embodiments, a graph cut algorithm is used to extract the main content. In short, graph cut (e.g., as applied in computer vision) is a segmentation technique used to segment an image into foreground and background elements. In graph theory, a cut is a partition of two disjoint subsets of the vertices of a graph (e.g., background and foreground elements). The output of image segmentation is a segmentation mask, which represents the specific pixel-wise boundary and shape of each category in the image (typically corresponding to different objects, features, or regions in the image). Different visual properties are used to indicate which category a particular pixel belongs to, such as the background or foreground category. Thus, in some embodiments, the segmentation map shows all background pixels as a first color, such as black, and all foreground pixels as one or more second colors.

[0061] In some embodiments, the grab-cut algorithm is implemented here as a subtype of the graph cut algorithm. Starting with the bounding box around the object to be segmented, the grab-cut algorithm uses a Gaussian mixture model to estimate the color distribution of the target object and the background. The color distribution is used to construct a Markov random field on the pixel labels, whose energy function preferentially connects regions with the same label. The grab-cut algorithm runs graph cut-based optimization to infer their values. This two-step process is repeated until convergence. If, in a particular iteration, more of the boundary regions around the object are identified as part of the object rather than part of the background, the size of the object has changed.

[0062] In some embodiments, the bounding box obtained at 306 (referred to as "rectangle C" for simplicity, and also as cluster box R2 as described above) is used as the initial region of interest (ROI) of the image, to which a graph cut algorithm is applied. Applying the graph cut algorithm to the ROI results in a segmented region, for example, referred to as "A1" for simplicity. The circumscribed region (referred to as "rectangle D" for simplicity) is calculated based on A1. Rectangle D is again used as the ROI, to which the graph cut algorithm is applied again. This process is repeated until a criterion is met, for example, the size of the graph cut region does not change from one iteration of graph cut to the next (or changes minimally according to a threshold value). Size refers to the size of an object within the bounding box, or the size of the bounding box surrounding the object without touching it. These iterations result in the final graph cut region (referred to as "A" for simplicity). For simplicity, the final graph cut region is also referred to as the "ROI grabbing box R3". The contents contained in the "ROI grabbing box R3" or "A" represent the segmented region obtained after the iterative graph cut process converges. The ROI grabbing box R3 or A is derived from the iterative refinement of the initial bounding box (R1). The final segmented region (ROI grabbing box R3 or A) can be a subset of the region covered by R1, for example, if the iterative refinement process identifies a more precise region contained within the initial bounding box R1. References to graph cut algorithms in some embodiments of this paper refer to grabbing and segmentation algorithms.

[0063] Rectangle C serves as the initial region of interest (ROI) obtained from clustering. In some embodiments, the size of Rectangle C may be smaller than other larger bounding boxes, such as Rectangle D or A1. Graph cut algorithms are specifically applied to this smaller region to refine the segmentation within the ROI. In some embodiments, the refined segmentation obtained after multiple iterations can extend the effective "coverage" of segmented objects within the initial ROI, effectively refining or expanding the region considered part of the object.

[0064] "Rotation features" refer to specific features or attributes within the content of a bounding box that indicate or inform information about the rotation of objects contained within the box. In some embodiments, rotation features can be extracted directly from the content within the bounding box, and rectangle C serves as a representation of cluster points within the box. Both rotation features and rectangle C can contribute to the accurate detection and adjustment of rotated objects. Rotation features act as auxiliary indicators to help estimate, adjust, and refine the detection of rotated objects within the bounding box. Rotation features provide visual cues and characteristics that enable the process to better understand and adapt to the rotation of objects, resulting in more accurate and reliable detection results.

[0065] Figure 5 This illustrates the use of graph cut algorithms to extract content regions in some embodiments (e.g., see reference). Figure 3The region of interest (ROI) is described at 306 (the bounding box). Initially, at 502, the graph cut algorithm is applied to the initial ROI region. In the first iteration, the initial ROI region is received as a clustering box R2 from the process shown in Figure 4 (e.g., as shown at 512). Applying the graph cut algorithm results in a segmented region (e.g., A1) at 506. The clustering box R2 represents the region in the image where the graph cut segmentation process can begin. The region at 514 represents the segmented region obtained after applying the graph cut algorithm to the initial ROI. The region at 514 shows the result of the iteration of the graph cut process. At 508, the circumscribed region (e.g., rectangle D) is calculated based on A1. The region at 516 represents the bounding box or region calculated based on the segmented region (A1) obtained from the graph cut algorithm. The region at 516 is derived from the segmented region and is used to define a new ROI for the next iteration. This region refers to the box that surrounds the identified object from the segmentation result / output. The box either touches the identified object or frames the identified object without touching the same object. At point 510, if the increase in ROI is greater than a threshold (e.g., 5%), the method iterates by proceeding to point 504, where the current ROI is used. Otherwise, the iteration ends. The processing at point 510 checks if the change (e.g., the increase in the ROI region (e.g., the difference between the current ROI and the previous ROI)) is greater than a predetermined threshold, such as 5%. If the increase is significant, the iteration continues by proceeding to point 504 and using the updated ROI. The same steps previously performed are then repeated, starting with the new ROI instead of the clustering box R2. If the increase in ROI is not significant, the iteration process ends. For example, the portion shown at point 518 represents the ROI of the last iteration or final iteration in a set of iterations.

[0066] Reference Figure 3 At 310, the method includes performing rotation and evaluation, such as rotating an initial bounding box and its contents according to a rotation angle determined based on a clustering box. In some embodiments, the rotation angle and size of the initial bounding box may be adjusted based on the clustering box and the region of interest (ROI) grasping box. For example, in some embodiments, the rotation and evaluation are performed using three image regions determined at 302, 304, and 306, such as the originally detected or initial bounding box (R1), the clustering bounding box (R2, also referred to as the clustering box), and the ROI grasping box (R3). Figure 6Examples of the original detected or initial bounding box (R1), cluster bounding box (R2, also called the clustering box), and ROI grabbing box (R3) in some embodiments are shown. In some embodiments, the rotation angle and size of the original detected or initial bounding box R1 602 can be refined based on R2 604 and R3 606. For example, the method finds a central axis (C-axis) 608 for R1 602 and attempts to repeatedly rotate the central axis 608 by an angle (e.g., as shown in the example at 610 and 612) until the points in R2 604 are divided into two parts, and the number of points on one side of the central axis is the same or substantially the same as the number of points on the other side of the central axis (plus or minus a threshold number). The rotation angle that divides the points in R2 into two equal or substantially equal parts is recorded as the target angle. Thus, the method utilizes the symmetry characteristics of the object to help identify the amount of rotation of the object orientation, such as the rotation angle.

[0067] The method adjusts the edges of R1 and calculates the Intersection over Union (IoU) of R3 and R1. For example, the method sets a step size value. This step size value can be pre-configured. The method reduces the left and right (e.g., lateral) edges by the step size value to obtain a new R1. The method calculates the IoU of the new R1 and R3. If the IoU increases, the method continues to adjust the top and bottom edges to obtain another new R1, then calculates the IoU again, and continues adjusting if the IoU increases. If the IoU decreases, the last adjustment is used as the target R1. After defining R2 and determining the rotation angle, R1 is adjusted, for example, as described above, further refining the bounding box to better align it with the segmented region represented by R3 (ROI grab box). By iteratively modifying the bounding box size based on the IoU metric, this adjustment finds the best-fit edges for R1, thereby improving the accuracy of bounding box localization around detected objects.

[0068] In some embodiments, sizing and rotation adjustments are performed together to optimize the initial bounding box modification. The optimal set of sizing and rotation angles is determined by automatically exploring combinations and finding the best pair. The optimal rotation angle may have a poor cross-union score. In some embodiments, these two values ​​are weighted equally for optimization. In some embodiments, one of these values, such as the rotation angle, is weighted more heavily for this optimization.

[0069] At 312, the method includes detecting objects within bounding boxes using a trained machine learning algorithm based on the rotation angle and size of the bounding box. In some embodiments, the method uses refined rotation angle and size parameters to narrow the region of interest for object detection. The trained machine learning algorithm then operates within that particular bounding box to identify and determine the presence and location of objects within that defined region. In some embodiments, an initial bounding box and its contents, such as outlined objects therein, are rotated according to a determined rotation angle. The rotated and / or resized initial bounding box and its contents are then fed into the trained machine learning algorithm. In some embodiments, only the rotated and / or resized initial bounding box is fed into the trained machine learning algorithm. In other embodiments, the entire initial image is fed into the trained machine learning algorithm as an image altered due to the rotation and / or resizing of the initial bounding box and its contents, while the remainder of the initial image remains unchanged. Rotation and resizing improve the accuracy of object detection by more precisely aligning the bounding box with the orientation and size of the actual object. In some embodiments, the method provides an overall workflow for performing rotated object detection. An optimized rotation generator based on candidate boxes and the contents of the boxes is provided. In some embodiments, the method involves using optimization techniques, such as graph cut techniques combined with machine learning algorithms, to iteratively adjust or generate the rotation of bounding boxes, with the aim of improving the accuracy of object detection, particularly for rotated objects within an image.

[0070] In some embodiments, this method allows for a reduction in the overall effort required to prepare data for machine learning or training in object detection. This method allows for the detection of rotated objects.

[0071] In some embodiments, the method does not require any modification to the training data. For example, the method obtains the rotation angle by correcting the detection region. From the perspective of rotation angle acquisition, the method can be completely unsupervised and does not require any rotation training data. The method simplifies rotated object detection by not requiring correction of the original training data, directly using the results of the detected bounding boxes, then correcting the results of the detected bounding boxes, and iteratively rotating the detected bounding boxes until the target can be optimally labeled.

[0072] Specific applications or use cases may include product location, classification, and / or counting, for example, for inventory or shelf replenishment in stores or warehouses. Being able to train an automatic object detection machine learning algorithm to detect rotating objects using raw training data including images reduces the amount of labeled training data needed to detect rotating objects appearing in images at various angles or viewpoints. Reducing the amount of data that needs to be processed frees up computational resources to perform other processing tasks, thereby increasing computational speed.

[0073] If the method described in this paper is applied to an object that is already axis-aligned and has not been rotated, the various steps described above are still performed, but the rotation angle and / or size adjustment of the initial bounding box is so small that it has no effect on the object detection performed by the machine learning algorithm (where the machine learning algorithm is trained using the axis-aligned image).

[0074] While some examples and figures illustrate tilted peripheral regions (e.g., portions of a shopping / goods box), the method described in this paper helps detect rotating objects that may be within such rotating peripheral regions. The method described in this paper for rotating portions of an image to the appropriate rotation angle rotates both the peripheral region of a square shopping box and the internal portions found within it (e.g., certain products or goods deployed within the square shopping box).

[0075] Figure 7 This diagram illustrates example components of a system capable of performing rotating object detection in some embodiments. One or more hardware processors 702, such as a central processing unit (CPU), graphics processing unit (GPU) and / or field-programmable gate array (FPGA), application-specific integrated circuit (ASIC), and / or other processors, may be coupled to storage device 704 and perform rotating object detection. Storage device 704 may include random access memory (RAM), read-only memory (ROM), or other storage devices, and may store data and / or processor instructions for implementing various functions associated with the methods and / or systems described herein. One or more processors 702 may execute computer instructions stored in storage device 704 or received from another computer device or medium. Storage device 704 may, for example, store instructions and / or data for the functions of one or more hardware processors 702, and may include operating system and other program instructions and / or data. Data used by one or more hardware processors 702 may be stored in storage device 706 or received from a remote device via network interface 708, and may be temporarily loaded into storage device 704 to perform rotating object detection. One or more hardware processors 702 may be coupled to an interface device such as a network interface 708 for communicating with a remote system, for example, via a network, and to an input / output interface 710 for communicating with input and / or output devices such as a keyboard, mouse, display, etc.

[0076] The terminology used herein is for the purpose of describing particular embodiments and is not intended to limit the invention. As used herein, the singular forms “a,” “an,” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. As used herein, the term “or” is an inclusive operator and may mean “and / or”, unless the context explicitly or clearly indicates otherwise. It will be further understood that, when used herein, the terms “comprising,” “including,” and / or “having” may specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. As used herein, the phrase “in some embodiments” does not necessarily refer to the same embodiment, although it may. As used herein, the phrase “in one embodiment” does not necessarily refer to the same embodiment, although it may. As used herein, the phrase “in another embodiment” does not necessarily refer to a different embodiment, although it may. Furthermore, embodiments and / or components of embodiments may be freely combined with each other unless they are mutually exclusive.

[0077] All means or steps plus functional elements (if any) in the following claims are intended to include corresponding structures, materials, actions, and equivalents for performing functions in combination with other elements specifically claimed, as specifically claimed. The invention has been described for illustrative and descriptive purposes, but is not intended to be exhaustive or to limit the invention to the forms disclosed. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the invention. Embodiments have been chosen and described to best explain the principles and practical application of the invention and to enable others skilled in the art to understand the invention in various embodiments and with various modifications for the particular application contemplated.

Claims

1. A computer-implemented method, comprising: Use neural networks to determine bounding boxes in an image; Extract the initial bounding boxes of objects in the image; Select the cluster centers of the bounding boxes and define the cluster boxes around the cluster centers; Rotate the initial bounding box and its contents according to the rotation angle determined based on the clustering box; as well as Use a trained machine learning model to detect objects within the rotated initial bounding box.

2. The computer-implemented method according to claim 1 further includes: The region of interest capture box is extracted by iteratively performing a graph cut algorithm starting with a clustered box as the initial region and obtaining segmented regions, where the segmented regions are used as the initial regions in the next iteration, and the iteration continues until the criteria are met; Adjust the size of the initial bounding box based on the region of interest capture box. The trained machine learning model is used to detect objects that occur within the initial bounding box after resizing.

3. The computer-implemented method according to claim 1 further includes training the neural network by using the crossover-union ratio as a mask for the true positive determination of the neural network's loss function.

4. The computer-implemented method of claim 1 further includes performing a non-maximum suppression process to select an initial bounding box from the bounding boxes.

5. The computer-implemented method of claim 1, wherein selecting the cluster center of the bounding box comprises determining representative pixels in each of the bounding boxes, clustering the representative pixels of the bounding boxes, and identifying the cluster of the highest density region with representative pixels.

6. The computer-implemented method according to claim 2, wherein the standard is satisfied if the size difference between the segmented region and the initial region is less than a predetermined threshold.

7. The computer-implemented method of claim 2, wherein the rotation angle of the initial bounding box is determined by finding the central axis of the initial bounding box, repeatedly rotating the central axis at an angle until the pixels in the cluster box are divided into two equal parts by the central axis, wherein the angle that divides the pixels in the cluster box into two equal parts by the central axis is used as the target rotation angle.

8. The computer-implemented method of claim 2, wherein adjusting the size of the initial bounding box includes adjusting the edges of the initial bounding box based on the intersection-union ratio of the region of interest capture box and the initial bounding box.

9. A computer program product, comprising: A collection of one or more computer-readable storage media; as well as Program instructions, which are collectively stored in a set of one or more storage media, are used to cause the processor set to perform the following computer operations: Use neural networks to determine bounding boxes in an image; Extract the initial bounding boxes of objects in the image; Select the cluster centers of the bounding boxes and define the cluster boxes around the cluster centers; Rotate the initial bounding box and its contents according to the rotation angle determined based on the clustering box; as well as Use a trained machine learning model to detect objects within the rotated initial bounding box.

10. The computer program product of claim 9, wherein the program instructions further cause the processor set to: Regions of interest (ROIs) are extracted by iteratively performing a graph cut algorithm, starting with clustered boxes as initial regions, to obtain segmented regions. These segmented regions are then used as initial regions in the next iteration, and the iteration continues until a criterion is met. Adjust the size of the initial bounding box based on the region of interest capture box. This involves the device using a trained machine learning model to detect objects with an initial bounding box that has been resized.

11. The computer program product of claim 9, wherein the program instructions further cause the processor set to train the neural network by using the intersection-union ratio as a mask for the true positive determination of the neural network as a loss function.

12. The computer program product of claim 9, wherein the program instructions further cause the processor set to perform a non-maximum suppression process to select an initial bounding box from the bounding boxes.

13. The computer program product of claim 9, wherein the program instructions further cause the processor set, when selecting the cluster center of the bounding box, to determine representative pixels in each of the bounding boxes, to cluster the representative pixels of the bounding boxes, and to identify the cluster of the highest density region with the representative pixels.

14. The computer program product of claim 10, wherein the standard is satisfied if the size difference between the segmented region and the initial region is less than a predetermined threshold.

15. The computer program product of claim 10, wherein the rotation angle of the initial bounding box is determined by finding the central axis of the initial bounding box, repeatedly rotating the central axis at an angle until the pixels in the cluster box are divided into two equal parts by the central axis, wherein the angle that divides the pixels in the cluster box into two equal parts by the central axis is used as the target rotation angle.

16. The computer program product of claim 10, wherein the program instructions further cause the processor set to adjust the edges of the initial bounding box based on the intersection-union ratio of the region of interest capture box and the initial bounding box when adjusting the size of the initial bounding box.

17. A computer system, comprising: Processor set; A collection of one or more computer-readable storage media; as well as Program instructions, which are collectively stored in a set of one or more storage media, are used to cause the processor set to perform the following computer operations: Use neural networks to determine bounding boxes in an image; Extract the initial bounding boxes of objects in the image; Select the cluster centers of the bounding boxes and define the cluster boxes around the cluster centers; Rotate the initial bounding box and its contents according to the rotation angle determined based on the clustering box; as well as Use a trained machine learning model to detect objects within the rotated initial bounding box.

18. The system of claim 17, wherein the program instructions further cause the processor set to: Regions of interest (ROIs) are extracted by iteratively performing a graph cut algorithm, starting with clustered boxes as initial regions, to obtain segmented regions. These segmented regions are then used as initial regions in the next iteration, and the iteration continues until a criterion is met. Adjust the size of the initial bounding box based on the region of interest capture box. This involves the device using a trained machine learning model to detect objects with an initial bounding box that has been resized.

19. The system of claim 17, wherein the program instructions further cause the processor set to train the neural network by using the intersection-union ratio as a mask for the true positive determination of the neural network as a loss function.

20. The system of claim 17, wherein the program instructions further cause the processor set to perform a non-maximum suppression process to select an initial bounding box from the bounding boxes.

Images