A target object rectangular frame extraction method and device, electronic equipment and medium
Patent Information
- Application Number
- CN202610777535.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-06-01
- Publication Date
- 2026-08-18
AI Technical Summary
然而，该方法易引入极其大量的背景区域，导致拟合精度极低
本申请实施例提供的一种目标对象矩形框的提取方法、装置、电子设备及介质，通过矩形框评价函数至少根据矩形框对目标对象的覆盖率以及目标对象在矩形框内的填充率对矩形框进行评分，最终确定该目标对象的目标矩形框，使得目标矩形框能够兼顾覆盖率和填充率，从而在目标对象的提取完整性和提取精度之间达到平衡。并且，覆盖率对评分的影响权重大于填充率对所述评分的影响权重，能够体现覆盖率优先的原则，符合人类视觉偏好，使得提取的目标矩形框更加合理。
Smart Images
Figure CN122597825A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the fields of computer vision and digital image processing technology, and in particular to a method, apparatus, electronic device and medium for extracting a rectangular bounding box of a target object. Background Technology
[0002] In computer vision applications, using bounding boxes to anchor target objects is a fundamental and crucial operation. For example, in scenarios such as robot vision navigation and object detection, extracting bounding boxes of obstacles can be used for subsequent contour extraction, navigation planning, and decision-making.
[0003] Existing mainstream bounding box extraction algorithms mainly fall into two categories: The first is extracting the minimum bounding rectangle (MinAreaRect), which aims to find the rectangle with the smallest area. However, this method is highly sensitive to noise, and in pursuit of the smallest area, it often removes irregular protruding parts of the target object's edges, resulting in incomplete target coverage. The second is extracting the horizontal bounding rectangle (BoundingBox), which only fits in the horizontal and vertical directions. However, this method easily introduces a large amount of background regions, leading to extremely low fitting accuracy. Therefore, existing bounding box extraction algorithms cannot meet the needs of practical engineering applications. Summary of the Invention
[0004] In view of this, the purpose of this application is to provide a method, apparatus, electronic device and medium for extracting a target object rectangle, so that the final determined target rectangle can take into account both coverage and fill rate, and achieve a balance between the completeness and accuracy of the target object extraction.
[0005] This application provides a method for extracting a rectangular bounding box of a target object, the method comprising: Acquire scene images of the target scene space through sensing devices; For each target object included in the scene image, multiple initial bounding boxes for that target object are generated in the scene image; Based on the plurality of initial bounding boxes, the target bounding box of the target object is determined by a predefined bounding box evaluation function; wherein, the bounding box evaluation function determines the score of the bounding box based at least on the coverage of the bounding box of the target object and the fill rate of the target object within the bounding box, and the influence weight of the coverage on the score is greater than the influence weight of the fill rate on the score.
[0006] Furthermore, the rectangular box evaluation function also determines the score of the rectangular box from the dimension of directional reward; the directional reward refers to applying a reward to the score of the rectangular box when the difference between the angle of the rectangular box and a preset angle is less than a preset threshold; the preset angle is determined based on prior knowledge of the distribution of the target object in the target scene space.
[0007] Furthermore, the rectangular box evaluation function is defined based on the coverage rate raised to the power of n1, the fill rate raised to the power of n2, and the directional reward raised to the power of n3; wherein n1, n2, and n3 are all positive integers, and n1 is greater than n2.
[0008] Furthermore, based on the plurality of initial bounding boxes, the target bounding box of the target object is determined by a predefined bounding box evaluation function, including: Taking the rectangle parameters of each initial rectangle as the starting point for optimization, the numerical gradient is calculated according to the rectangle evaluation function, and the rectangle parameters are iteratively updated using the gradient descent method. When the preset conditions are met, the target rectangle parameters are obtained. The target rectangle of the target object is generated based on the target rectangle parameters.
[0009] Furthermore, taking the rectangle parameters of each initial rectangle as the starting point for optimization, the numerical gradient is calculated according to the rectangle evaluation function, and the rectangle parameters are iteratively updated using the gradient descent method. When preset conditions are met, the target rectangle parameters are obtained, including: A perturbation is applied to the rectangular parameters of the initial bounding box, and the rate of change of the score of the bounding box evaluation function before and after the perturbation is determined to obtain the numerical gradient; The rectangle parameters are updated using gradient descent based on the numerical gradient, resulting in the updated rectangle parameters. Based on the updated rectangle parameters and the rectangle evaluation function, determine whether the gradient update has stalled; If stagnation occurs, the rectangle parameters of the initial rectangle are actively adjusted according to a preset strategy to obtain updated rectangle parameters; Return to the step of applying a perturbation to the rectangle parameters of the initial rectangle to iteratively update the rectangle parameters until the target rectangle parameters are obtained when the preset conditions are met.
[0010] Furthermore, for each target object within the scene image, multiple initial bounding boxes are generated for that target object, including: For each target object, at least one bounding box extraction algorithm is used to generate at least one first initial bounding box for the target object; For each first initial rectangle, perform rectangle angle perturbation and / or rectangle ratio perturbation to generate multiple second initial rectangles for the target object; Using the center of each first and second initial rectangle as the origin, the area of the rectangle is perturbed to generate multiple third initial rectangles for the target object. Each first initial rectangle, each second initial rectangle, and each third initial rectangle are scored based on the rectangle evaluation function. Select the top N rectangles with the highest scores as multiple initial rectangles for the target object; where N is a positive integer.
[0011] Furthermore, before generating multiple initial bounding boxes for each target object within the scene image, the extraction method further includes: The scene image is preprocessed by morphological filtering and hole filling.
[0012] This application embodiment also provides a device for extracting a rectangular frame of a target object, the extraction device comprising: The acquisition module is used to acquire scene images of the target scene space through sensing devices; The first generation module is used to generate multiple initial bounding boxes for each target object within the scene image. The second generation module is used to determine the target rectangle of the target object based on the plurality of initial rectangles using a predefined rectangle evaluation function; wherein the rectangle evaluation function determines the score of the rectangle based at least on the coverage of the rectangle on the target object and the fill rate of the target object within the rectangle, and the influence weight of the coverage on the score is greater than the influence weight of the fill rate on the score.
[0013] This application embodiment also provides an electronic device, including: a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor communicates with the memory via the bus. When the machine-readable instructions are executed by the processor, the steps of the target object rectangle extraction method described above are performed.
[0014] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, performs the steps of the target object rectangle extraction method described above.
[0015] The embodiments of this application have the following beneficial technical effects: This application provides a method, apparatus, electronic device, and medium for extracting a target object's bounding box. The method uses a bounding box evaluation function to score the bounding box based on at least two factors: the coverage rate of the bounding box over the target object and the fill rate of the target object within the bounding box. This ultimately determines the target bounding box of the target object, achieving a balance between coverage and fill rate in the extraction process. Furthermore, the influence of coverage on the score is greater than that of fill rate, reflecting a principle of prioritizing coverage and aligning with human visual preferences, resulting in a more reasonable extracted target bounding box.
[0016] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description
[0017] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.
[0018] Figures 1(a) and (b) show schematic diagrams of the extraction effect of the rectangular box extraction algorithm in the prior art; Figure 2 A flowchart illustrating a method for extracting a target object rectangle according to an embodiment of this application is shown; Figure 3 This diagram illustrates the extraction effect of the rectangular frame provided in an embodiment of this application. Figure 4 This illustration shows a schematic diagram of the structure of a target object rectangle extraction device provided in an embodiment of this application; Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of this application is shown. Detailed Implementation
[0019] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. The components of the embodiments of this application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely represents selected embodiments of this application. Based on the embodiments of this application, every other embodiment obtained by those skilled in the art without inventive effort falls within the scope of protection of this application.
[0020] Research has shown that using bounding boxes to anchor target objects is a fundamental and crucial operation in computer vision applications. For example, in scenarios such as robot vision navigation and object detection, extracting bounding boxes of obstacles can be used for subsequent contour extraction, navigation planning, and decision-making.
[0021] Please refer to Figures 1(a) and (b), which are schematic diagrams illustrating the extraction effect of the rectangular box extraction algorithm in the prior art provided in the embodiments of this application. Existing mainstream rectangular box extraction algorithms mainly include two types: one is extracting the minimum bounding rectangle (MinAreaRect), aiming to find the rectangle with the smallest area. However, this method is very sensitive to noise, and in pursuit of the smallest area, it often cuts off irregular protruding parts of the target object's edge, resulting in incomplete target coverage, as shown by the green rectangular boxes in Figures 1(a) and (b). The other is extracting the horizontal bounding rectangle (BoundingBox), fitting only in the horizontal and vertical directions. However, this method easily introduces an extremely large amount of background area, resulting in extremely low fitting accuracy, as shown by the blue rectangular boxes in Figures 1(a) and (b). Therefore, existing rectangular box extraction algorithms cannot meet the needs of practical engineering applications.
[0022] Based on this, embodiments of this application provide a method for extracting the bounding box of a target object, so that the final determined bounding box can take into account both coverage and fill rate, achieving a balance between the completeness and accuracy of the extracted target object.
[0023] First, the applicable scenarios of this application are introduced. This application can be applied to the fields of computer vision and digital image processing technology, and is particularly suitable for scenarios such as sensor data processing, robot visual navigation, and target detection. In one example, the extraction method can be performed by a robot. Different types of robots need to identify target objects in the scene space for downstream applications when performing different types of tasks. For example, when a cleaning robot performs a cleaning task in a car showroom, it needs to identify car targets in the showroom and extract the expected vehicle outline information for subsequent navigation planning and obstacle avoidance.
[0024] Please see Figure 2 , Figure 2 This is a flowchart illustrating a method for extracting a rectangular bounding box of a target object, as provided in an embodiment of this application. Figure 2 As shown in the embodiments of this application, the extraction method includes: S201. Acquire scene images of the target scene space through sensing devices.
[0025] Corresponding to the example above, sensors mounted on the cleaning robot can be used as sensing devices to acquire scene images of the target environment where the robot performs its cleaning tasks. Subsequent steps can then extract bounding boxes from these scene images to identify the car target. These scene images can be point cloud images.
[0026] S202. For each target object included in the scene image, generate multiple initial bounding boxes for that target object in the scene image.
[0027] Here, the target object can be defined according to actual technical requirements. For example, for a cleaning robot that moves autonomously to perform cleaning tasks or a delivery robot that performs delivery tasks, the target object can be an obstacle in the target scene space on the path of movement; for a handling robot, the target object can also be an item that needs to be moved or grasped.
[0028] In this step, for each target object included in the scene image, multiple initial bounding boxes of the target object can be generated and labeled in the scene image using existing algorithms (such as the aforementioned minimum bounding rectangle, horizontal bounding rectangle, etc.).
[0029] S203. Based on the plurality of initial rectangles, determine the target rectangle of the target object using a predefined rectangle evaluation function.
[0030] The rectangular box evaluation function determines the score of the rectangular box based at least on the coverage of the target object and the fill rate of the target object within the rectangular box, and the influence weight of the coverage on the score is greater than the influence weight of the fill rate on the score.
[0031] Here, the coverage of the target object by the rectangle refers to the proportion of the target object that the rectangle can cover, reflecting the completeness of the target object extraction; the fill rate of the target object within the rectangle refers to the proportion of the target object within the rectangle, reflecting the extraction accuracy of the target object. Referring to Figures 1(a) and (b), the coverage is the number of white pixels within the rectangle / the total number of white pixels in the connected components of the target object; the fill rate is the number of white pixels within the rectangle / the total number of pixels in the rectangle.
[0032] In this step, a score can be determined for each initial rectangle using a rectangle evaluation function. The score is positively correlated with the extraction performance; a higher score indicates better extraction of the target object and makes it more likely to be selected as the final target rectangle. Accordingly, based on the scores of each initial rectangle, at least one initial rectangle with the highest score is selected as the target rectangle for the target object.
[0033] Furthermore, in the rectangular box evaluation function, the influence weight of coverage on the score is greater than that of fill rate; that is, under the same change, the increase or decrease in score caused by the change in coverage will be greater than the increase or decrease in score caused by the change in fill rate, meaning that the change in score is more sensitive to the change in coverage.
[0034] In this way, the determined target bounding boxes can balance coverage and fill rate, thus achieving a balance between the completeness and accuracy of target object extraction. Furthermore, the influence of coverage on the score is weighted greater than that of fill rate, reflecting the principle of prioritizing coverage, conforming to human visual preferences, and making the extracted target bounding boxes more reasonable and suitable for downstream applications.
[0035] Furthermore, the rectangular box evaluation function also determines the score of the rectangular box from the dimension of directional reward; the directional reward refers to applying a reward to the score of the rectangular box when the difference between the angle of the rectangular box and a preset angle is less than a preset threshold; the preset angle is determined based on prior knowledge of the distribution of the target object in the target scene space.
[0036] Here, a preset angle can be determined based on prior knowledge of the distribution of the target object in the target scene space. When the difference between the angle of the rectangle and the preset angle is less than a preset threshold, a reward is applied to the score of the rectangle to avoid outputting a non-intuitive slightly tilted result as the target rectangle. This encourages the rectangle's angle to converge quickly to the preset angle, thereby improving the efficiency and accuracy of rectangle extraction.
[0037] Corresponding to the example above, when the cleaning robot performs cleaning tasks in a car showroom, the cars are generally parked in a regular pattern, meaning the car body should be horizontally aligned with the walls of the target scene space. Accordingly, based on prior knowledge of this distribution, a preset angle of 0° or 180° is set to avoid slightly tilted rectangles. Simultaneously, this directional reward is applied with the precondition that the difference between the rectangle's angle and the preset angle is less than a preset threshold, thus preventing the erroneous correction of truly tilted target objects (with a difference greater than the preset threshold) to the preset angle.
[0038] Mathematically, the rectangular box evaluation function can be a linearly weighted form; preferably, the rectangular box evaluation function can also be defined based on the coverage rate raised to the power of n1, the fill rate raised to the power of n2, and the directional reward raised to the power of n3; where n1, n2, and n3 are all positive integers, and n1 is greater than n2. Generally, n2 is also greater than n3, which better reflects the non-linear relationship of coverage priority among the evaluation dimensions.
[0039] In one example, the formula for the rectangle evaluation function is expressed as:
[0040] in, Represents the rectangular box evaluation function; Indicates coverage rate; Indicates fill rate; Indicates directional reward.
[0041] More specifically:
[0042] The coverage exponent n1 is set to 4.0 (extremely high sensitivity), meaning that even a small decrease in coverage will result in a drastic drop in score. For example, if the coverage drops from 1.0 to 0.9, the score drops to 0.9^4 ≈ 0.65, thus forcing the algorithm to expand the rectangle as much as possible to encompass the target object. The fill rate exponent n2 is set to 2.0 (medium sensitivity), used to constrain the rectangle from expanding indefinitely and maintain a moderate level of compactness. The directional reward exponent n3 is set to 1.0.
[0043] In one possible implementation, step S202 may include: S2021. For each target object, at least one first initial rectangle is generated for that target object using at least one rectangle extraction algorithm.
[0044] In this step, the horizontal bounding rectangle of the target object can be generated based on the prior knowledge that most objects have a horizontal orientation, and used as the first initial rectangle; alternatively, the minimum bounding rectangle (minAreaRect) containing the OpenCV standard can be used as the first initial rectangle.
[0045] S2022. Perform rectangle angle perturbation and / or rectangle ratio perturbation on each first initial rectangle to generate multiple second initial rectangles for the target object.
[0046] In this step, each first initial rectangle can be perturbed by a small angle, such as rotating it ±2° around the center of the rectangle as the origin; and / or, each first initial rectangle can be perturbed by a rectangle ratio, generated according to different ratios (such as aspect ratio 1.5 ~ 3.5).
[0047] S2023. Using the center of the rectangle of each first initial rectangle and second initial rectangle as the origin, perturb the area of the rectangle to generate multiple third initial rectangles for the target object.
[0048] In this step, the area of the rectangle is perturbed. The center of the rectangle can be used as the origin to generate indented rectangles with areas reduced to different proportions such as 80% and 90%.
[0049] Here, the obtained first, second, and third initial rectangles can be directly used as initial rectangles; alternatively, initial rectangles can be further filtered from them. Specifically: S2024. Scoring is performed on each first initial rectangle, each second initial rectangle, and each third initial rectangle based on the rectangle evaluation function.
[0050] S2025. Select the top N rectangles with the highest scores as multiple initial rectangles for the target object.
[0051] Where N is a positive integer. For example, there are 30 initial rectangles in total, including the first, second, and third initial rectangles, from which the first 15 are selected as the initial rectangles.
[0052] Through the above methods, the embodiments of this application can generate a wide range of initial rectangles, increasing the likelihood that they contain the optimal rectangle and avoiding the omission of the best solution.
[0053] Furthermore, embodiments of this application can also determine the target rectangle of the target object through iterative optimization based on multiple initial rectangles and a rectangle evaluation function. In this implementation, the original solution of the initial rectangle algorithm can be used to generate a rich set of initial solutions as the starting point for optimization. This also prevents the optimization algorithm from getting trapped in local optima (e.g., trapped in a pure rectangle within a local part of an object), helping to obtain the optimal rectangle as the target rectangle.
[0054] Furthermore, prior to S202, the extraction method in this embodiment of the application further includes: preprocessing the scene image by morphological filtering and hole filling.
[0055] Here, the scene image is a binarized image; for morphological filtering, a 10x10 pixel rectangular kernel can be used for closing operations. This step, dilation followed by erosion, connects minor breaks around the main object without significantly altering its outline size, avoiding the mistaken merging of independent obstacles as with large kernel dilation. For hole filling, all closed white connected regions are retrieved, and their internal black holes are replaced with white. This step eliminates the interference of internal noise on subsequent rectangle fitting (statistical fill rate). Here, morphological closing operations are used to connect the broken target regions and perform hole filling to construct solid connected regions.
[0056] The specific method of obtaining the target rectangle through iterative optimization in the embodiments of this application will be described below.
[0057] In one possible implementation, step S203 may include: S2031. Taking the rectangle parameters of each initial rectangle as the starting point for optimization, calculate the numerical gradient according to the rectangle evaluation function, and use the gradient descent method to iteratively update the rectangle parameters. When the preset conditions are met, the target rectangle parameters are obtained.
[0058] In this process, the rectangle parameters of each initial rectangle are combined into a multi-dimensional vector, such as... ,in, and These represent the x and y coordinates of the center of the rectangle, respectively. Indicates the width of the rectangle; Indicates the height of the rectangle; This indicates the rotation angle of the rectangle.
[0059] Since the scene image is a discrete pixel grid, an analytical solution cannot be obtained. This embodiment applies a small perturbation to each parameter in the multidimensional vector. Calculate the rectangular box evaluation function The rate of change is used to obtain an approximate gradient. Then, the rectangle parameters are iteratively updated using gradient descent methods, such as momentum gradient descent.
[0060] The preset conditions can be that the iteration reaches a certain number of times or that the solution is determined to be optimal. The specific conditions can be set as needed, and this application embodiment does not impose any restrictions.
[0061] S2032. Generate the target rectangle frame of the target object according to the target rectangle parameters.
[0062] In this way, after obtaining the target rectangle parameters, the target rectangle of the target object can be generated in the scene image according to the set of parameters.
[0063] In specific implementation, step S2031 may include: Step 1: Apply a perturbation to the rectangle parameters of the initial rectangle, determine the rate of change of the score of the rectangle evaluation function before and after applying the perturbation, and obtain the numerical gradient.
[0064] Step 2: Update the rectangle parameters using gradient descent based on the numerical gradient to obtain the updated rectangle parameters.
[0065] Step 3: Determine whether the gradient update has stalled based on the updated rectangle parameters and the rectangle evaluation function.
[0066] Step 4: If a standstill occurs, the rectangle parameters of the initial rectangle are actively adjusted according to a preset strategy to obtain updated rectangle parameters.
[0067] Regarding steps 3 and 4, to prevent gradient descent from getting stuck in local optima, this application provides an active shrinkage strategy. When gradient updates are determined to have stalled, the rectangle parameters of the initial bounding box are actively adjusted according to a preset strategy. Preferably, the rectangle boundary can be tightened from the outside in, effectively eliminating unnecessary black blank spaces at the rectangle edges by simulating the shrinking process of a rubber band. For example, actively attempting to reduce the length or width of the rectangle by a fixed step size (such as 1px, 2px, 5px) allows the algorithm to escape the current solution range and explore other parameter spaces.
[0068] Next, return to the step of perturbing the rectangle parameters of the initial bounding box, i.e., step 1 mentioned above, to iteratively update the rectangle parameters. If the rectangle obtained after active shrinking has a slightly lower coverage but an increased fill rate, it will lead to a higher score. If the value increases, the change from this proactive adjustment can be accepted, and iterative optimization can continue according to the parameters of the rectangle after proactive shrinkage, until the target rectangle parameters are obtained when the preset conditions are met.
[0069] Or, if rating If the value decreases significantly, the proactive adjustment can be rejected, the rectangle parameters can be restored to their original state, and the strategy can be adjusted, such as changing the parameters or the step size. The score can then be determined again. Alternatively, a maximum number of executions can be set for the proactive contraction strategy. This iterative optimization continues until the target rectangle parameters are obtained when the preset conditions are met.
[0070] Please see Figure 3 , Figure 3 This is a schematic diagram illustrating the extraction effect of the rectangular frame provided in the embodiments of this application; as shown Figure 3 As shown, the target bounding boxes R1 and R2 generated by the extraction method provided in this application avoid the situation where the target subject is cut off in pursuit of rectangular purity, which is more in line with the intuitive human definition of a bounding box; it also avoids the situation where an extremely large background area is introduced in pursuit of complete coverage, resulting in extremely low fitting accuracy. That is, a balance is achieved between the completeness and accuracy of the extracted target object.
[0071] This application provides a method for extracting a target object bounding box, comprising: acquiring a scene image of a target scene space through a sensing device; generating multiple initial bounding boxes for each target object included in the scene image; determining a target bounding box for the target object based on the multiple initial bounding boxes using a predefined bounding box evaluation function; wherein the bounding box evaluation function determines the bounding box score based at least on the coverage rate of the bounding box to the target object and the fill rate of the target object within the bounding box, and the influence weight of the coverage rate on the score is greater than the influence weight of the fill rate on the score.
[0072] In this way, the bounding box evaluation function scores the bounding box based on at least two factors: the coverage rate of the bounding box to the target object and the fill rate of the target object within the bounding box. This ultimately determines the target bounding box for the target object, achieving a balance between coverage and fill rate in the extraction process, thus balancing the completeness and accuracy of the extracted target object. Furthermore, the influence of coverage on the score is greater than that of fill rate, reflecting the principle of prioritizing coverage and aligning with human visual preferences, resulting in a more reasonable extracted target bounding box.
[0073] Based on the same inventive concept, embodiments of this application also provide a device for extracting a rectangular frame of a target object. Please refer to... Figure 4 , Figure 4 This is a schematic diagram of a device for extracting a rectangular frame of a target object, provided in an embodiment of this application. Figure 4 As shown, the extraction device 400 includes: The acquisition module 410 is used to acquire scene images of the target scene space through the sensing device; The first generation module 420 is used to generate multiple initial bounding boxes for each target object in the scene image. The second generation module 430 is used to determine the target rectangle of the target object based on the plurality of initial rectangles by a predefined rectangle evaluation function; wherein the rectangle evaluation function determines the score of the rectangle based at least on the coverage of the rectangle on the target object and the fill rate of the target object within the rectangle, and the influence weight of the coverage on the score is greater than the influence weight of the fill rate on the score.
[0074] Furthermore, the rectangular box evaluation function also determines the score of the rectangular box from the dimension of directional reward; the directional reward refers to applying a reward to the score of the rectangular box when the difference between the angle of the rectangular box and a preset angle is less than a preset threshold; the preset angle is determined based on prior knowledge of the distribution of the target object in the target scene space.
[0075] Furthermore, the rectangular box evaluation function is defined based on the coverage rate raised to the power of n1, the fill rate raised to the power of n2, and the directional reward raised to the power of n3; wherein n1, n2, and n3 are all positive integers, and n1 is greater than n2.
[0076] Furthermore, when the second generation module 430 determines the target rectangle of the target object based on the plurality of initial rectangles using a predefined rectangle evaluation function, the second generation module 430 is used to: Taking the rectangle parameters of each initial rectangle as the starting point for optimization, the numerical gradient is calculated according to the rectangle evaluation function, and the rectangle parameters are iteratively updated using the gradient descent method. When the preset conditions are met, the target rectangle parameters are obtained. The target rectangle of the target object is generated based on the target rectangle parameters.
[0077] Furthermore, when the second generation module 430 uses the rectangle parameters of each initial rectangle as the starting point for optimization, calculates the numerical gradient according to the rectangle evaluation function, and iteratively updates the rectangle parameters using the gradient descent method, and obtains the target rectangle parameters when the preset conditions are met, the second generation module 430 is used to: A perturbation is applied to the rectangular parameters of the initial bounding box, and the rate of change of the score of the bounding box evaluation function before and after the perturbation is determined to obtain the numerical gradient; The rectangle parameters are updated using gradient descent based on the numerical gradient, resulting in the updated rectangle parameters. Based on the updated rectangle parameters and the rectangle evaluation function, determine whether the gradient update has stalled; If stagnation occurs, the rectangle parameters of the initial rectangle are actively adjusted according to a preset strategy to obtain updated rectangle parameters; Return to the step of applying a perturbation to the rectangle parameters of the initial rectangle to iteratively update the rectangle parameters until the target rectangle parameters are obtained when the preset conditions are met.
[0078] Furthermore, when the first generation module 420 generates multiple initial bounding boxes for each target object within the scene image, the first generation module 420 is used to: For each target object, at least one bounding box extraction algorithm is used to generate at least one first initial bounding box for the target object; For each first initial rectangle, perform rectangle angle perturbation and / or rectangle ratio perturbation to generate multiple second initial rectangles for the target object; Using the center of each first and second initial rectangle as the origin, the area of the rectangle is perturbed to generate multiple third initial rectangles for the target object. Each first initial rectangle, each second initial rectangle, and each third initial rectangle are scored based on the rectangle evaluation function. Select the top N rectangles with the highest scores as multiple initial rectangles for the target object; where N is a positive integer.
[0079] Furthermore, the extraction device 400 also includes a preprocessing module; the preprocessing module is used for: The scene image is preprocessed by morphological filtering and hole filling.
[0080] Please see Figure 5 , Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 5 As shown, the electronic device 500 includes a processor 510, a memory 520, and a bus 530.
[0081] The memory 520 stores machine-readable instructions that can be executed by the processor 510. When the electronic device 500 is running, the processor 510 and the memory 520 communicate via the bus 530. When the machine-readable instructions are executed by the processor 510, the steps of the target object rectangle extraction method as described in the above method embodiment can be performed. For specific implementation details, please refer to the method embodiment, which will not be repeated here.
[0082] This application also provides a computer-readable storage medium storing a computer program. When the computer program is run by a processor, it can execute the steps of the target object rectangle extraction method as described in the above method embodiments. For specific implementation details, please refer to the method embodiments, which will not be repeated here.
[0083] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.
[0084] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Additionally, the shown or discussed mutual couplings, direct couplings, or communication connections may be through some communication interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms.
[0085] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.
[0086] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.
[0087] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a processor-executable, non-volatile, computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
[0088] Finally, it should be noted that the above-described embodiments are merely specific implementations of this application, used to illustrate the technical solutions of this application, and not to limit them. The scope of protection of this application is not limited thereto. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some of the technical features, within the scope of the technology disclosed in this application. Such modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.
Claims
1. A method for extracting a rectangular bounding box of a target object, characterized in that, The method includes: Acquire scene images of the target scene space through sensing devices; For each target object included in the scene image, multiple initial bounding boxes for that target object are generated in the scene image; Based on the plurality of initial bounding boxes, the target bounding box of the target object is determined by a predefined bounding box evaluation function; wherein, the bounding box evaluation function determines the score of the bounding box based at least on the coverage of the bounding box of the target object and the fill rate of the target object within the bounding box, and the influence weight of the coverage on the score is greater than the influence weight of the fill rate on the score.
2. The extraction method according to claim 1, characterized in that, The bounding box evaluation function also determines the score of the bounding box from the dimension of directional reward; the directional reward refers to applying a reward to the score of the bounding box when the difference between the angle of the bounding box and the preset angle is less than the preset threshold; the preset angle is determined based on prior knowledge of the distribution of the target object in the target scene space.
3. The extraction method according to claim 2, characterized in that, The rectangular box evaluation function is defined based on the coverage rate raised to the power of n1, the fill rate raised to the power of n2, and the directional reward raised to the power of n3; wherein n1, n2, and n3 are all positive integers, and n1 is greater than n2.
4. The extraction method according to claim 1, characterized in that, Based on the plurality of initial bounding boxes, the target bounding box of the target object is determined by a predefined bounding box evaluation function, including: Taking the rectangle parameters of each initial rectangle as the starting point for optimization, the numerical gradient is calculated according to the rectangle evaluation function, and the rectangle parameters are iteratively updated using the gradient descent method. When the preset conditions are met, the target rectangle parameters are obtained. The target rectangle of the target object is generated based on the target rectangle parameters.
5. The extraction method according to claim 4, characterized in that, Starting with the rectangle parameters of each initial rectangle, the numerical gradient is calculated based on the rectangle evaluation function, and the rectangle parameters are iteratively updated using gradient descent. When preset conditions are met, the target rectangle parameters are obtained, including: A perturbation is applied to the rectangular parameters of the initial bounding box, and the rate of change of the score of the bounding box evaluation function before and after the perturbation is determined to obtain the numerical gradient; The rectangle parameters are updated using gradient descent based on the numerical gradient, resulting in the updated rectangle parameters. Based on the updated rectangle parameters and the rectangle evaluation function, determine whether the gradient update has stalled; If stagnation occurs, the rectangle parameters of the initial rectangle are actively adjusted according to a preset strategy to obtain updated rectangle parameters; Return to the step of applying a perturbation to the rectangle parameters of the initial rectangle to iteratively update the rectangle parameters until the target rectangle parameters are obtained when the preset conditions are met.
6. The extraction method according to claim 1, characterized in that, For each target object within the scene image, generate multiple initial bounding boxes for that target object, including: For each target object, at least one bounding box extraction algorithm is used to generate at least one first initial bounding box for the target object; For each first initial rectangle, perform rectangle angle perturbation and / or rectangle ratio perturbation to generate multiple second initial rectangles for the target object; Using the center of each first and second initial rectangle as the origin, the area of the rectangle is perturbed to generate multiple third initial rectangles for the target object. Each first initial rectangle, each second initial rectangle, and each third initial rectangle are scored based on the rectangle evaluation function. Select the top N rectangles with the highest scores as multiple initial rectangles for the target object; where N is a positive integer.
7. The extraction method according to claim 1, characterized in that, Before generating multiple initial bounding boxes for each target object within the scene image, the extraction method further includes: The scene image is preprocessed by morphological filtering and hole filling.
8. A device for extracting a rectangular frame of a target object, characterized in that, The extraction device includes: The acquisition module is used to acquire scene images of the target scene space through sensing devices; The first generation module is used to generate multiple initial bounding boxes for each target object within the scene image. The second generation module is used to determine the target rectangle of the target object based on the plurality of initial rectangles using a predefined rectangle evaluation function; wherein the rectangle evaluation function determines the score of the rectangle based at least on the coverage of the rectangle on the target object and the fill rate of the target object within the rectangle, and the influence weight of the coverage on the score is greater than the influence weight of the fill rate on the score.
9. An electronic device, characterized in that, include: The device includes a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor communicates with the memory via the bus. The machine-readable instructions are executed by the processor to perform the steps of the target object rectangle extraction method as described in any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, performs the steps of the method for extracting the target object rectangle as described in any one of claims 1 to 7.