A method and apparatus for training a recognition model

Abstract

This application provides a method and apparatus for training a recognition model to improve image annotation efficiency and thereby optimize the recognition model. Specifically, the method may include: acquiring a first image, wherein the first image may include a first semantic object and a second semantic object; automatically annotating the boundary information between the first semantic object and the second semantic object based on the positional relationship between them, and updating the bounding box of the first semantic object. This effectively improves the efficiency of image annotation, thereby optimizing the accuracy and effectiveness of the recognition model.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a method and apparatus for training a recognition model. Background Technology

[0002] In fields such as autonomous driving, smart healthcare, and smart cities, deep learning-based recognition models are needed to identify target objects in application scenarios. These recognition models require training on a large amount of labeled sample data.

[0003] However, the sample data corresponding to this recognition model is mostly images. The current recognition model cannot accurately identify the boundaries between different semantic objects in the image (e.g., the boundary between vehicles and pedestrians), the boundary of the area where a specific semantic object (e.g., road surface) is located in the image, and the boundary of the area formed by the isolation of a specific semantic object (e.g., green belt) in the image. A lot of manual annotation is required, and there are also cases of repeated annotation between the boundaries of different semantic objects, which leads to the problem of low annotation efficiency for such sample data.

[0004] Therefore, how to improve the efficiency of labeled data and obtain a recognition model with high accuracy is the technical problem that this application aims to solve. Summary of the Invention

[0005] This application provides a method and apparatus for training a recognition model, which improves the efficiency of image annotation and thereby optimizes the recognition model.

[0006] In a first aspect, embodiments of this application provide a training method for a recognition model. The method includes: first, acquiring a first image, wherein the first image may include at least two semantic objects, specifically including a first semantic object and a second semantic object; then, updating the bounding box of the first semantic object according to a first positional relationship (e.g., external tangency, internal tangency, intersection, etc.) between the first bounding box of the first semantic object and the second bounding box of the second semantic object, to obtain an updated first bounding box of the first semantic object, wherein the updated first bounding box and the second bounding box have the same first target directed line segment; finally, generating a second image based on the first image, the second bounding box, and the updated first bounding box, wherein the second image contains a boundary marker that can be used to indicate the first target directed line segment; and inputting the second image into a recognition model for training to obtain a trained recognition model.

[0007] In this embodiment, based on the positional relationship between the first semantic object and the second semantic object, the boundary information between the first semantic object and the second semantic object can be automatically labeled, and the bounding box of the first semantic object can be updated, thereby effectively improving the efficiency of labeled images and achieving the effect of obtaining labeled sample data at low cost and high efficiency, thereby optimizing the accuracy and effectiveness of the recognition model.

[0008] The first bounding box that needs to be updated can be preset or selected by the user. This application embodiment does not impose specific restrictions.

[0009] In one possible design, a first operation can be received before updating the bounding box of the first semantic object; based on the first operation, the first bounding box is determined to be the first layer, the second bounding box is determined to be the second layer, and the first target directed line segment is located on the second layer.

[0010] In this design, users can select the semantic objects whose bounding boxes need to be updated according to their actual needs, thereby obtaining labeled images that meet their requirements and effectively improving the user experience.

[0011] There are multiple ways to obtain the first image in the embodiments of this application, which can be obtained based on virtual data or real data.

[0012] Method 1: Obtain the bounding box of at least one semantic object, and randomly combine the bounding boxes of at least one semantic object to obtain the first image.

[0013] It should be understood that "random combination" refers to placing the bounding boxes of different semantic objects together. This application does not impose specific limitations on the positional relationships between each semantic object in the image generated after "random combination." For example, by randomly combining the complete bounding boxes corresponding to common road objects in a vehicle driving environment, such as vehicles, pedestrians, and green belts, a vehicle driving environment image can be obtained. In this image, the bounding box of a vehicle can be tangent to the bounding box of a green belt, and disjoint from the bounding box of a pedestrian.

[0014] Method 1 saves manpower and material resources, and can quickly obtain the first image, thereby effectively improving the efficiency of subsequent acquisition of labeled samples.

[0015] Method 2 involves acquiring a third image (i.e., real data) and preprocessing it using a preset algorithm (object detection algorithm) to obtain the approximate outline of each semantic object in the third image. Then, the outline of each semantic object is manually corrected, and the complete bounding box corresponding to each semantic object is marked to obtain the first image.

[0016] Method 2 ensures the accuracy of the first image, thereby making the subsequent labeled images more accurate.

[0017] In one possible design, the process of updating the bounding box of the first semantic object based on the first positional relationship between the first bounding box of the first semantic object and the second bounding box of the second semantic object may specifically include: Step 1, determining at least two intersection points of the first and second bounding boxes; Step 2, based on the at least two intersection points, splitting the first bounding box into at least two first directed line segments and splitting the second bounding box into at least two second directed line segments; Step 3, based on the first positional relationship, determining the first directed line segments located outside the second bounding box and determining the second directed line segments located inside the first bounding box; Step 4, determining the updated first bounding box based on the first directed line segments located outside the second bounding box and the second directed line segments located inside the first bounding box. Each first directed line segment and each second directed line segment includes a start point and an end point.

[0018] In this design, the updated first bounding box can be obtained simply by considering the positional relationship between the first directed line segment after the first bounding box is split and the second bounding box, as well as the positional relationship between the second directed line segment after the second bounding box is split and the first bounding box. The calculation is simple and efficient, effectively improving the annotation efficiency of the boundary between the first and second bounding boxes.

[0019] In one possible design, the updated first bounding box is determined based on the first directed line segment located outside the second bounding box and the second directed line segment located inside the first bounding box. Specifically, the updated first bounding box can be obtained by connecting the first directed line segment located outside the second bounding box and the second directed line segment located inside the first bounding box.

[0020] In this design, automatic annotation of the first bounding box is achieved, thereby improving the annotation efficiency of the first image.

[0021] In one possible design, when it is determined based on the first positional relationship that there is a second directed line segment located on the first bounding box among at least two second directed line segments, then the second directed line segment is determined to be a second directed line segment located inside the first bounding box; when it is determined based on the first positional relationship that there is a first directed line segment located on the second bounding box among at least two first directed line segments, then the first directed line segment is determined to be a first directed line segment located inside the second bounding box.

[0022] In this design, by flexibly setting the rules for determining the relationship between line segments and bounding boxes, the first bounding box updated based on the line segments after the bounding box is split is more accurate, effectively improving the accuracy of image annotation.

[0023] The process of determining at least two intersection points of the first and second bounding boxes in step 1 can be implemented in different ways for different line segments.

[0024] 1. Intersecting line segments.

[0025] In one possible design, determining at least two intersection points of the first bounding box and the second bounding box includes: determining that any first directed line segment is not parallel to any second directed line segment and that a first intersection point exists; determining that the first intersection point lies on any first directed line segment and any second directed line segment; and if the first intersection point is not the endpoint of any first directed line segment and not the endpoint of any second directed line segment, then determining the first intersection point as any one of the at least two intersection points.

[0026] In this design, the intersection of the first and second bounding boxes is only identified when the intersection point of the intersecting line segments does not fall at the end of the line segment. This effectively avoids the situation where the intersection point between the first and second bounding boxes is determined repeatedly.

[0027] In one possible design, determining that the first intersection point lies on either a first directed line segment or a second directed line segment includes: determining that the distance between the first intersection point and either first directed line segment is less than a first threshold, thus determining that the first intersection point lies on either first directed line segment; and / or, determining that the distance between the first intersection point and either second directed line segment is less than a second threshold, thus determining that the first intersection point lies on either second directed line segment.

[0028] In this design, the distance between the intersection point and the line segment is determined by approximation, which effectively improves the flexibility of intersection point setting and makes the intersection point determination more accurate. This, in turn, makes the line segments obtained by dividing based on the first and second bounding boxes of the intersection point more accurate.

[0029] In one possible design, at least two intersection points of the first bounding box and the second bounding box are determined. If any first directed line segment and any second directed line segment have a second intersection point, and the distance between the second intersection point and the first intersection point is less than a third threshold, then the second intersection point and the first intersection point are considered to be the same intersection point.

[0030] This design further enhances the flexibility of intersection point setting, thereby reducing the segment division of the first and second bounding boxes, effectively improving the efficiency of updating the first bounding box, and thus improving the efficiency of image annotation.

[0031] 2. Parallel line segments.

[0032] In one possible design, determining at least two intersection points of the first and second bounding boxes may include: determining that any first directed line segment is parallel to any second directed line segment; any first directed line segment includes a first starting point and a first ending point, and any second directed line segment includes a second starting point and a second ending point; determining that any first directed line segment coincides with any second directed line segment if the distance between the first starting point or the first ending point and any second directed line segment is less than a fourth threshold; or, determining that any first directed line segment coincides with any second directed line segment if the distance between the second starting point or the second ending point and any first directed line segment is less than a fifth threshold; and determining the first starting point and / or the second starting point as any of the at least two intersection points.

[0033] In this design, considering that the coordinate values ​​of each point on any first directed line segment and any second directed line segment are irrational numbers, the overlap between any first directed line segment and any second directed line segment is determined by approximation, and the overlapping line segments are flexibly determined, thereby making the intersection points determined based on the overlapping line segments more accurate.

[0034] In one possible design, defining the first starting point and / or the second starting point as any of at least two intersection points includes: defining the second starting point as any intersection point of the first bounding box and the second bounding box when the second starting point is on any of the first directed line segments; or defining the first starting point as any intersection point of the first bounding box and the second bounding box when the first starting point is on any of the second directed line segments.

[0035] In this design, the second or second starting point is only set as the intersection of the first and second bounding boxes after it is determined that the first or second starting point falls on a coincident line segment. This further improves the accuracy of the determined intersection points between coincident line segments.

[0036] In one possible design, at least two semantic objects also include a third semantic object; after obtaining the updated first bounding box of the first semantic object, it can be determined that the third bounding box of the third semantic object is located within the first bounding box; and when it is determined that the third bounding box of the third semantic object is internally tangent to the updated first bounding box, the bounding box of the first semantic object is updated again to obtain the first bounding box of the first semantic object after a second update; wherein the third bounding box and the first bounding box after a second update have the same second target directed line segment; based on the first image, the second bounding box, the third bounding box, and the first bounding box after a second update, a fourth image is generated; based on the fourth image, the trained recognition model is retrained to obtain a new recognition model.

[0037] In this design, if the positional relationship between the first bounding box and the bounding boxes of other semantic objects changes (e.g., becomes inscribed), the first bounding box is updated a second time. This can improve the accuracy of the boundary annotation of semantic objects with multiple positional relationships, effectively improve the accuracy of the labeled samples, and thus optimize the recognition model.

[0038] In one possible design, when the first positional relationship is intersecting, the updated first bounding box includes a first sub-bounding box and a second sub-bounding box. That is, the first bounding box is truncated by the second bounding box, where "truncated" can be understood as the first bounding box being occluded by the second bounding box. Both the first and second sub-bounding boxes share common boundary segments with the second bounding box.

[0039] In this design, the first bounding box can be automatically updated through the first positional relationship, realizing automatic annotation of the first bounding box in occluded scenes, thereby improving the efficiency of annotation.

[0040] In one possible design, at least two semantic objects also include a third semantic object; after obtaining the updated first bounding box of the first semantic object, if it is determined that the first sub-bounding box and / or the second sub-bounding box intersects with the third bounding box of the third semantic object, the bounding box of the first semantic object is updated again to obtain the second updated first bounding box; wherein, the second updated first bounding box includes three sub-bounding boxes.

[0041] In this design, for scenarios where the first bounding box is occluded by multiple semantic objects, the boundary can be automatically labeled, effectively improving image labeling efficiency.

[0042] Secondly, embodiments of this application provide a training method for a recognition model. The method includes: first, acquiring a first image, the first image including at least two semantic objects, the at least two semantic objects including a first semantic object and a second semantic object; second, determining the bounding box of a blank area in the first image based on the first bounding box of the first semantic object and the second bounding box of the second semantic object; then, generating a second image based on the first image, the second bounding box, the first bounding box, and the bounding box of the blank area; finally, inputting the second image into the recognition model for training to obtain the trained recognition model. The bounding box of the blank area shares the same third target directed line segment as the first bounding box, and the bounding box of the blank area shares the same fourth target directed line segment as the second bounding box.

[0043] In this embodiment, the bounding boxes of the blank areas can be determined by the bounding boxes of the semantic objects already marked in the first image (e.g., the first bounding box and the second bounding box), which effectively improves the annotation efficiency of the blank areas in the first image, thereby effectively improving the efficiency of obtaining sample data and achieving the effect of optimizing the accuracy of the recognition model.

[0044] Thirdly, embodiments of this application provide a training method for a recognition model. The method includes: first, acquiring a first image, the first image including at least one semantic object, and the at least one semantic object including the first semantic object; second, determining the bounding box of an isolated region in the first image based on the first bounding box of the first semantic object and the boundary of the first image; then, generating a second image based on the first image, the first bounding box, and the bounding box of the isolated region; and finally, inputting the second image into the recognition model for training to obtain the trained recognition model. The isolated region can be understood as a region that does not intersect with the bounding box of at least one semantic object; the bounding box of the isolated region includes at least two sub-bounding boxes, and any one of the at least two sub-bounding boxes has the same third target directed line segment as the first bounding box, and / or has the same fourth target directed line segment as the second bounding box.

[0045] In this embodiment, the bounding boxes of the semantic objects already marked in the first image can be used to determine the bounding boxes of the blank areas, which can effectively improve the annotation efficiency of isolated areas in the first image, thereby improving the efficiency of obtaining sample data and achieving the effect of optimizing the accuracy of the recognition model.

[0046] Fourthly, embodiments of this application provide a training apparatus for a recognition model. For example, the apparatus may include:

[0047] An acquisition module is used to acquire a first image; wherein the first image includes at least two semantic objects, and the at least two semantic objects include a first semantic object and a second semantic object;

[0048] The processing module is used to update the bounding box of the first semantic object according to the first positional relationship between the first bounding box of the first semantic object and the second bounding box of the second semantic object, so as to obtain the updated first bounding box of the first semantic object; wherein, the first positional relationship includes any one of external tangency, internal tangency, and intersection, and the updated first bounding box and the second bounding box have the same first target directed line segment; generate a second image based on the first image, the second bounding box, and the updated first bounding box; input the second image into the recognition model for training to obtain the trained recognition model.

[0049] In addition, other alternative implementations of the communication device in this regard can be found in the relevant content of the first aspect above, and will not be described in detail here.

[0050] Fifthly, embodiments of this application provide a training apparatus for a recognition model. For example, the apparatus may include:

[0051] An acquisition module is used to acquire a first image, the first image including at least two semantic objects, the at least two semantic objects including a first semantic object and a second semantic object;

[0052] The processing module is used to determine the bounding box of the blank area in the first image based on the first bounding box of the first semantic object and the second bounding box of the second semantic object; wherein the bounding box of the blank area has the same third target directed line segment as the first bounding box and the same fourth target directed line segment as the second bounding box; generate a second image based on the first image, the second bounding box, the first bounding box, and the bounding box of the blank area; input the second image into the recognition model for training to obtain the trained recognition model.

[0053] In addition, other alternative implementations of the communication device in this regard can be found in the relevant content of the second aspect above, and will not be described in detail here.

[0054] Sixthly, embodiments of this application provide a training apparatus for a recognition model. Exemplarily, the apparatus may include:

[0055] An acquisition module is used to acquire a first image, the first image including at least one semantic object, and the at least one semantic object including the first semantic object;

[0056] The processing module is configured to determine the bounding box of an isolated region in the first image based on the first bounding box of the first semantic object and the boundary of the first image; wherein the isolated region is a region that does not intersect with the bounding box of at least one semantic object; the bounding box of the isolated region includes at least two bounding boxes, any one of the at least two bounding boxes having the same third target directed line segment as the first bounding box, and / or having the same fourth target directed line segment as the second bounding box; generate a second image based on the first image, the first bounding box, and the bounding box of the isolated region; and input the second image into the recognition model for training to obtain the trained recognition model.

[0057] In addition, other alternative implementations of the communication device in this regard can be found in the relevant content of the third aspect above, and will not be described in detail here.

[0058] In a seventh aspect, embodiments of this application provide a vehicle that may include a processor for performing the methods described in the first, second, or third aspects described above, as well as any possible designs of the first, second, or third aspects described above.

[0059] Eighthly, embodiments of this application provide a server including a processor for performing the methods described in the first, second, or third aspects above, as well as any possible designs of the first, second, or third aspects above.

[0060] In one possible design, the server is a single server or a server cluster consisting of multiple sub-servers. When the server is a server cluster consisting of multiple sub-servers, the multiple sub-servers jointly execute the methods described in the first aspect, the second aspect, or the third aspect above, as well as any possible design of the first aspect, the second aspect, or the third aspect above.

[0061] Ninthly, embodiments of this application provide a chip system including at least one processor, wherein when program instructions are executed in the at least one processor, the methods described in the first aspect, the second aspect, or the third aspect, and any of the optional designs of the first aspect, the second aspect, or the third aspect, are implemented.

[0062] In one possible design, the chip system also includes a communication interface for inputting or outputting information.

[0063] In one possible design, the chip system also includes a memory coupled to the processor via a communication interface for storing the aforementioned instructions, so that the processor can read the instructions stored in the memory via the communication interface.

[0064] In one possible design, the processor described above can be a processing circuit, which is not limited in this application.

[0065] In a tenth aspect, embodiments of this application also provide a computer program product including instructions, which, when run on the aforementioned apparatus, performs a method for training an identification model as described in the first aspect or any possible design of the first aspect, the second aspect or any possible design of the second aspect, or the third aspect or any possible design of the third aspect.

[0066] Eleventhly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when run, implements a training method for an identification model as described in the first aspect or any possible design of the first aspect, the second aspect or any possible design of the second aspect, or the third aspect or any possible design of the third aspect.

[0067] For the beneficial effects of aspects four through eleven above, please refer to the technical effects that can be achieved by the corresponding designs in aspects one through three above, which will not be repeated here. Attached Figure Description

[0068] Figure 1 A scenario illustration provided for this application;

[0069] Figure 2 A schematic diagram of an image annotation provided for this application;

[0070] Figure 3 This is a schematic diagram of the deployment of the model training device provided in the embodiments of this application;

[0071] Figure 4 A flowchart illustrating a training method for a recognition model provided in an embodiment of this application;

[0072] Figure 5 A schematic diagram illustrating a possible scenario of a positional relationship provided for an embodiment of this application;

[0073] Figure 6 A schematic diagram of intersecting line segments provided in an embodiment of this application;

[0074] Figure 7 A schematic diagram of a scenario involving parallel line segments provided in an embodiment of this application;

[0075] Figure 8 A scenario diagram provided for an embodiment of this application;

[0076] Figure 9 This is another scenario illustration provided for an embodiment of this application;

[0077] Figure 10 This is another scenario illustration provided for an embodiment of this application;

[0078] Figure 11 A flowchart illustrating a training method for a recognition model provided in an embodiment of this application;

[0079] Figure 12A A scenario diagram provided for an embodiment of this application;

[0080] Figure 12B This is another scenario illustration provided for an embodiment of this application;

[0081] Figure 13 A flowchart illustrating a training method for a recognition model provided in an embodiment of this application;

[0082] Figure 14A A scenario diagram provided for an embodiment of this application;

[0083] Figure 14B This is another scenario illustration provided for an embodiment of this application;

[0084] Figure 15A schematic diagram of the structure of a training device for a recognition model provided in an embodiment of this application;

[0085] Figure 16 A schematic diagram of the structure of a training device for another recognition model provided in an embodiment of this application;

[0086] Figure 17 A schematic diagram of the structure of a training device for another recognition model provided in an embodiment of this application;

[0087] Figure 18 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0088] The following explanations of some terms used in the embodiments of this application are provided to facilitate understanding by those skilled in the art.

[0089] 1) A bounding box can be understood as a fictional outer frame surrounding each semantic object in an image to be labeled, or the outer frame of a specific image region in the image to be labeled. In digital image processing, a bounding box can be the coordinates of the polygonal border that completely surrounds the digital image when it is placed on a page, canvas, screen, or other similar two-dimensional background. Simply put, a bounding box is a geometric image with simple characteristics used to approximate complex semantic objects.

[0090] 2) Blank areas can be understood as the image regions corresponding to specific semantic objects set in the image to be labeled in a specific application scenario, such as the image regions where the road or sky is located in an autonomous driving scenario.

[0091] 3) An isolated region can be understood as an image region in the image to be labeled that is isolated by one or more semantic objects in a specific application scenario, such as the image region of the road surface isolated by the green belt in an autonomous driving scenario.

[0092] 4) In the embodiments of this application, "at least one" refers to one or more, and "at least two" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can represent: a, b, c, a and b, a and c, b and c, or a, b, and c, where a, b, and c can be single or multiple. Furthermore, unless otherwise stated, in the embodiments of this application, ordinal numbers such as "first" and "second" are used to distinguish multiple objects and are not used to limit the size, shape, content, order, timing, priority, or importance of multiple objects. For example, the first bounding box and the second bounding box are only used to distinguish different bounding boxes, and do not indicate that the two bounding boxes are different in size, shape, priority or importance.

[0093] The preceding text introduced some terms and concepts involved in the embodiments of this application. The following text introduces the technical features involved in the embodiments of this application.

[0094] In recent years, deep learning has permeated various fields of machine vision (such as autonomous driving, smart healthcare and smart cities). Based on deep learning recognition models, target objects in different application scenarios can be identified, such as the boundaries of target objects.

[0095] Among them, the recognition model based on supervised deep learning (the process of training an intelligent algorithm to map input data to labels using known data and their one-to-one correspondences) has the best recognition performance. However, supervised deep learning requires a large amount of labeled sample data for training to obtain a recognition model with high accuracy. Therefore, obtaining a large amount of accurately labeled sample data is particularly important.

[0096] As described above, the sample data for recognition models in machine vision are mostly image data. Current recognition models cannot accurately identify the boundaries between different semantic objects in an image (e.g., the boundary between vehicles and pedestrians), the boundaries of regions containing specific semantic objects (e.g., road surfaces), and the boundaries of regions isolated by specific semantic objects. This requires extensive manual annotation, resulting in inefficiency. For example, ... Figure 1The semantic objects in the image to be labeled shown in (a) include object A and object B, bounding box A corresponding to object A and bounding box B corresponding to object B. The current recognition model cannot accurately identify the boundary between bounding box A and bounding box B, and manual annotation is required to obtain it.

[0097] To annotate the boundaries between semantic objects in an image, one approach involves painting the pixel region corresponding to each semantic object in the image the same color, and then calculating the boundary coordinates of the painted pixel region using a preset algorithm. This yields the bounding box of each semantic object and the boundaries between different semantic objects. However, this approach requires constant adjustments to the brush size when depicting pixel regions of semantic objects with rich details or uneven boundaries (e.g., buildings), which is time-consuming and still results in low annotation efficiency.

[0098] In another implementation: the boundaries of semantic objects in the image to be labeled are manually drawn using a pen or polygon drawing tool to obtain the pixel region corresponding to each semantic object in the image to be labeled. Then, the labeled pixels in each pixel region are marked as 1, and the unlabeled pixels are marked as 0. An image mask composed of 0s and 1s is generated for each pixel region. Specifically, if a pixel in the pixel region corresponding to the semantic object to be labeled in the image has been labeled as 1 by another semantic object, then the position of that pixel is marked as 0. In this way, the boundary labeling between different semantic objects can be completed.

[0099] However, this scheme requires converting the pixel region boundaries corresponding to each semantic object into a mask representation. If there are many semantic objects in the image to be labeled, and they are very small, there will be a lot of redundant calculations, which is very time-consuming and still results in low labeling efficiency. Furthermore, masks can only handle integer coordinates, while the coordinates of pixel region boundaries obtained by drawing tools or using a pen are coordinates with precise decimals. Therefore, using masks for image labeling has a large precision error, which leads to the boundary between different semantic objects not coinciding with the actual boundary. Figure 1 As shown in (a), the semantic objects in the image to be labeled are object A and object B. The bounding box A corresponding to object A is occluded by the bounding box B corresponding to object B. The boundary between bounding boxes A and B obtained using this technique is... Figure 1 Line segment 1 is shown in (b), however, line segment 1 does not coincide with the actual boundary.

[0100] In another implementation: the image to be labeled is segmented into multiple superpixel blocks (small regions composed of a series of adjacent pixels with similar features such as color, brightness, and texture) using a preset algorithm; superpixel blocks with the same semantic information (e.g., color) are merged into one class; then, the merged superpixel blocks are labeled one by one according to user operations until the boundary of each semantic object in the image to be labeled is obtained (e.g., color). Figure 2 The shapes of circles, hexagons, and pentagrams in the text), and the boundaries between different semantic objects (e.g., circles, hexagons, pentagrams, etc.). Figure 2 (The boundary between circles and hexagons in the image). However, this scheme requires very high precision in the superpixel algorithm. The higher the precision, the smaller the superpixel block, and the more complex the operation. It is difficult to control the granularity of pixel block merging. If the merging granularity is too large, some semantic objects with smaller pixel regions cannot be labeled, resulting in poor image labeling accuracy.

[0101] To address the aforementioned technical problems, embodiments of this application provide a training method and apparatus for a recognition model, which automatically labels a first image, enabling the acquisition of highly accurate labeled sample data at low cost and high efficiency, and then training a highly accurate recognition model based on the sample data.

[0102] It should be understood that in the embodiments of this application, the first image is a pre-processed image to be labeled, and the complete bounding box of each semantic object in the first image has been labeled. The technical solution of the embodiments of this application can automatically label the boundaries between different semantic objects based on the first positional relationship between different semantic objects (e.g., intersection, internal tangency, etc.), which can effectively improve the efficiency of labeled samples.

[0103] It should be noted that the training method of the recognition model in the embodiments of this application can be applied to any field of machine vision. For example:

[0104] In the medical field, the image to be labeled can be an image of a certain type of lesion, such as an image of a leg fracture in a patient with a leg fracture. The technical solution of this application embodiment can automatically label the common boundaries between various semantic objects in the image of a leg fracture, obtain labeled images, and then train a recognition model based on the labeled images, so that the recognition model can be used to identify new images of leg fractures. The boundaries between various semantic objects in the obtained images of leg fractures are clearer, which effectively improves the recognition accuracy of fracture patients and reduces the workload of manually identifying fracture patients.

[0105] In the field of autonomous driving, the image to be labeled can be an image of the vehicle's driving environment. The technical solution of this application embodiment can automatically label the boundaries between different semantic objects in the image (e.g., the boundary between a vehicle and a building), blank areas, and isolated areas. This yields more accurate labeled samples and effectively improves the efficiency of data labeling. The labeled samples are then used to optimize and train the recognition model, effectively improving the accuracy of the vehicle in recognizing object boundaries during autonomous driving, thus facilitating the completion of autonomous driving.

[0106] The following section uses the field of autonomous driving as an example to introduce the system architecture applicable to the embodiments of this application.

[0107] In the field of autonomous driving, the training method for the recognition model provided in this application can be specifically applied to vehicle-to-everything (V2X), long-term evolution to vehicle (LTE-V), and vehicle-to-vehicle (V2V) communication. For example, it can be applied to vehicles with image processing capabilities, or to other components in vehicles with image processing capabilities. These other components include, but are not limited to, on-board terminals, on-board controllers, on-board modules, on-board components, on-board chips, on-board units, on-board radar, or on-board cameras, and other sensors. The vehicle can implement the training method for the recognition model provided in this application through these on-board terminals, on-board controllers, on-board modules, on-board components, on-board chips, on-board units, on-board radar, or cameras.

[0108] Of course, the training method of the recognition model in this embodiment can also be applied to other smart terminals with image processing functions besides vehicles, or installed in other smart terminals with image processing functions besides vehicles, or installed in components of such smart terminals. The smart terminal can be other terminal devices such as smart transportation equipment, smart home devices, robots, etc. Examples include, but are not limited to, smart terminals or controllers, chips, radar or cameras, and other sensors and components within smart terminals.

[0109] In the field of autonomous driving, the training method for the recognition model provided in this application embodiment can also be applied to a vehicle-to-everything (V2X) server. This V2X server is a device with image processing capabilities, which can be a physical device, such as a host or server, or a virtual device, such as a virtual machine or container. It should be noted that, for ease of description, the V2X server will be simply referred to as a server below. That is, in the specific implementation process, the term "server" as used below can refer to a server in the common sense, or it can refer to other devices with image processing capabilities, or it can refer to a module within a device (such as a chip or integrated circuit).

[0110] Specifically, the training method for the recognition model provided in this application embodiment can be executed by a model training device, and the location of the model training device is not limited in this application embodiment. For example, as shown... Figure 3 As shown, the model training device can run on a cloud computing device system (including at least one cloud computing device, such as a vehicle server), or on an edge computing device system (including at least one edge computing device, such as a server, desktop computer, etc.), or on various terminal computing devices, such as in-vehicle terminals, laptops, personal desktop computers, etc.

[0111] It should be noted that the above-mentioned model training device has different processing methods for different image regions in the image to be labeled (such as the boundary region between different semantic objects, blank regions, and isolated regions). Therefore, in order to clearly illustrate the training method of the recognition model provided in the embodiments of this application, the training method of the recognition model in the embodiments of this application will be described with reference to the accompanying drawings, taking three types of image regions as examples.

[0112] Please see Figure 4 To automatically annotate the boundary regions between different semantic objects in an image to be annotated, this application provides a method for training a recognition model, the method comprising:

[0113] S401: Obtain the first image.

[0114] The first image includes at least two semantic objects, which are a first semantic object and a second semantic object. These at least two semantic objects can be understood as image data corresponding to at least two semantic objects in the first image, such as the first bounding box of the first semantic object and the second bounding box of the second semantic object. It should be understood that the first image is the image to be labeled mentioned above, and the complete bounding box of each semantic object in the first image has been marked, for example... Figure 5 The first bounding box of the first semantic object and the second bounding box of the second semantic object are shown in (b).

[0115] This application does not limit the source of the first image; it can be obtained by processing real data or by processing virtual data. Real data can be understood as images in the actual environment, and virtual data can be understood as image data obtained through preset rules.

[0116] The following sections will introduce the processing methods for real data and virtual data respectively:

[0117] I. Real Data

[0118] In one possible implementation, a third image (i.e., real data) can be acquired and preprocessed using a preset algorithm (object detection algorithm) to obtain the approximate outline of each semantic object in the third image. Then, the outline of each semantic object is manually corrected, and the complete bounding box corresponding to each semantic object is annotated to obtain the first image. This ensures the accuracy of the first image, thereby making the subsequently annotated images more accurate.

[0119] There are several ways to obtain the third image, including but not limited to the following:

[0120] Method 1: Use images of the vehicle's driving environment captured by the vehicle's onboard camera as the third image. This effectively reduces the manpower cost of acquiring the third image.

[0121] Method 2 involves obtaining point clouds of multiple objects (e.g., road surface, vehicles, pedestrians, etc.) in the vehicle's driving environment using radar installed on the vehicle. These point clouds are then processed through clustering and point cloud envelope generation, and finally projected onto a two-dimensional plane to obtain a third image. This reduces the human cost of acquiring the third image while retaining features closer to the actual environment, thus improving the accuracy of the third image.

[0122] Method 3 uses images of the vehicle's driving environment downloaded from the network or taken by staff as a third image. This reduces hardware costs.

[0123] It should be understood that the above methods can be used in combination or individually, and the embodiments of this application do not limit this.

[0124] II. Virtual Data

[0125] In one possible implementation, bounding boxes of at least one semantic object (e.g., vehicle, pedestrian, green belt, etc.) can be obtained from a cloud device or a local server. These bounding boxes are then randomly combined to obtain a first image (i.e., virtual data). This eliminates the need to capture real images of the vehicle's driving environment or to preprocess those images, saving manpower and resources and allowing for rapid acquisition of the first image, thereby significantly improving the efficiency of subsequent sample annotation.

[0126] It should be understood that "random combination" refers to placing the bounding boxes of different semantic objects together. This application does not specifically limit the positional relationships between each semantic object in the image generated after "random combination." For example, by randomly combining the complete bounding boxes corresponding to common road objects in a driving environment such as vehicles, pedestrians, and green belts, a driving environment image containing the bounding boxes of vehicles, pedestrians, and green belts can be obtained. In this image, the bounding box of a vehicle can be tangent to the bounding box of the green belt and disjoint from the bounding box of a pedestrian; or, in this image, the bounding box of a vehicle can intersect with the bounding box of the green belt and be tangent to the bounding box of a pedestrian; or, in this image, the bounding box of a vehicle can intersect with both the bounding box of the green belt and the bounding box of a pedestrian. See details for further information. Figure 5 Taking the first bounding box of the first semantic object and the second bounding box of the second semantic object as an example, the random combination of the first bounding box and the second bounding box can result in the following: Figure 5 Middle (b) to Figure 5 The first bounding box and the second bounding box shown in (e) constitute the first image.

[0127] It should be understood that the two methods of acquiring the first image described above can be implemented individually or in combination.

[0128] S402: Based on the first positional relationship between the first bounding box of the first semantic object and the second bounding box of the second semantic object, update the bounding box of the first semantic object to obtain the updated first bounding box of the first semantic object.

[0129] It should be noted that each semantic object in the first image may exhibit self-occlusion. For an example, please refer to [link to example image]. Figure 5 In Figure 5(a), the first bounding box of the first semantic object exhibits self-occlusion (a part of the object occludes another part of the object, making the occluded part invisible). For first bounding boxes with self-occlusion, the occluded part is transformed to obtain a first bounding box without self-occlusion, and then subsequent update processing is performed. In this way, the accuracy of the first bounding box obtained in subsequent updates is effectively improved.

[0130] In this application embodiment, the first positional relationship may specifically include one or more of external tangency, internal tangency, and intersection. When the first positional relationship is internal tangency, it may be internal tangency between the first semantic object and the second semantic object, or internal tangency between the second semantic object and the first semantic object. This application does not impose any restrictions.

[0131] For example, please continue to see Figure 5 ,like Figure 5 As shown in (b), when the first positional relationship is intersection, the first bounding box and the second bounding box intersect; as Figure 5 As shown in (c), when the first positional relationship is externally tangent, the first bounding box and the second bounding box are externally tangent; as Figure 5 As shown in (d), when the first positional relationship is inscribed, the first bounding box and the second bounding box are inscribed together, or, as shown in (d), Figure 5 As shown in (e), the second bounding box is internally tangent to the first bounding box. It should be noted that, in this embodiment of the application, there is a boundary between the first bounding box and the second bounding box. Figure 5 When the inscribed relationship is shown in (e), the first bounding box can be updated; when there is a relationship between the first bounding box and the second bounding box... Figure 5 When the inscribed relationship is shown in (d), the second bounding box can be updated.

[0132] It should be understood that in the embodiments of this application, the updated first bounding box and the second bounding box have the same first target directed line segment, wherein the "first target directed line segment" can be understood as the same boundary line segment in the updated first bounding box and the second bounding box.

[0133] In one possible implementation, the bounding box that needs to be updated in the first image can be preset as the first bounding box. Then, a bounding box of a semantic object with complete boundary annotations can be randomly selected from the first image as the second bounding box. The first bounding box is then updated based on the positional relationship between the first and second bounding boxes. This implementation can improve image processing efficiency.

[0134] In another possible implementation, the bounding box that needs to be updated in the first image can be determined as the first bounding box, and the second bounding box used to update the first bounding box can be determined based on the user's operation, thereby determining the layer in which the first bounding box is located.

[0135] For example: Before updating the bounding box of the first semantic object, a first operation is received; based on the first operation, the first bounding box is determined to be the first layer, the second bounding box to be the second layer, and the first target directed line segment is located on the second layer. Here, the first layer is below the second layer. The "first layer" is the layer containing the target semantic object (i.e., the first semantic object) whose bounding box needs to be updated in the first image, and the "second layer" is the layer containing the semantic object (i.e., the second semantic object) whose boundary is shared in the first image. The "first operation" refers to an operation performed by the user. For example, when the technical solution provided in this application is applied to a data annotation platform, the user operation detected in the user interface provided by the data annotation platform can specifically be an operation to input the first semantic object identifier and the second semantic object identifier, or an operation where the user clicks on the first semantic object identifier. This application does not impose specific limitations.

[0136] The second image obtained based on this implementation method can better meet user needs.

[0137] In one possible implementation, updating the bounding box of the first semantic object based on the first positional relationship between the first bounding box of the first semantic object and the second bounding box of the second semantic object may include the following steps 1 to 4:

[0138] Step 1: Determine at least two intersection points of the first and second bounding boxes.

[0139] It should be noted that the division of the first and second bounding box segments in step 2 depends on the intersection point of the two bounding boxes. Therefore, the accuracy of the intersection point determines the accuracy of the bounding box segment division, and consequently, the accuracy of the updated first bounding box. Thus, determining a reasonable intersection point is extremely important. One way to determine the intersection point is to calculate it using a preset geometric algorithm. Since the calculated coordinates of the intersection point are irrational numbers and deviate somewhat from the actual positions, a flexible determination of the intersection point is necessary to ensure its accuracy.

[0140] It should be understood that the intersection point between two bounding boxes is the intersection point between the directed line segments of the two bounding boxes. Because the directed line segments within the bounding boxes of different semantic objects have different positional relationships (e.g., intersecting line segments, parallel line segments, etc.), the method for determining the intersection point between two line segments differs depending on their positional relationship.

[0141] The following examples illustrate how to determine the intersection point of line segments with different positional relationships.

[0142] A. For intersecting line segments, determine that any first directed line segment in the first bounding box and any second directed line segment in the second bounding box are not parallel and have a first intersection point. Since the endpoint of each directed line segment is the starting point of the next line segment, to avoid duplicate intersection points, if the first intersection point falls on the endpoint of a directed line segment, it is not considered an intersection point. Therefore, the first intersection point is determined as any one of at least two intersection points only when it is determined that the first intersection point lies on either a first or second directed line segment, and the first intersection point is not the endpoint of either a first or second directed line segment. This effectively avoids the situation where duplicate intersection points are determined between the first and second bounding boxes.

[0143] For example, please see Figure 6 If any first directed line segment is taken as line segment 1, and any second directed line segment is taken as line segment 2, and line segment 1 and line segment 2 intersect, and line segment 1 includes start point 1 and end point 1, and line segment 2 includes start point 2 and end point 2, and line segment 1 and line segment 2 have an intersection point 1, and the intersection point 1 is not the end point 1 and end point 2, then the intersection point 1 is determined as the intersection point of the first bounding box and the second bounding box.

[0144] Furthermore, since the coordinates of intersection point 1 are irrational numbers, in actual coordinates, intersection point 1 may not fall on line segment 1 or line segment 2. Therefore, if the distance between intersection point 1 and line segment 1 is less than the first threshold, then intersection point 1 is considered to be on line segment 1; and if the distance between intersection point 1 and line segment 2 is less than the second threshold, then intersection point 1 is considered to be on line segment 2. This effectively improves the flexibility of intersection point setting, making the intersection point determination more accurate, and the line segments obtained by dividing based on the first and second bounding boxes of the intersection point are more accurate.

[0145] In one possible implementation, line segment 1 and line segment 2 may have two intersection points, such as intersection point 1 and intersection point 2. When the distance between intersection point 1 and intersection point 2 is less than a third threshold, intersection point 1 and intersection point 2 can be considered as the same intersection point. This further improves the flexibility of intersection point setting, thereby reducing the segment division of the subsequent first and second bounding boxes and effectively improving the efficiency of updating the bounding boxes.

[0146] B. For parallel line segments, if any first directed line segment of the first bounding box is parallel and coincident with any second directed line segment of the second bounding box, then the starting point of any first directed line segment or any second directed line segment is determined as the intersection point of the first and second bounding boxes. This flexible setting of the intersection point can reduce subsequent line segment division operations, thereby improving the efficiency of updating the first bounding box.

[0147] Optionally, when the second starting point of any second directed line segment lies on any first directed line segment, the second starting point is determined as the intersection of the first bounding box and the second bounding box; or, when the first starting point of any first directed line segment lies on any second directed line segment, the first starting point is determined as the intersection of the first bounding box and the second bounding box. This further improves the accuracy of the determined intersection points between coincident line segments.

[0148] For example, please see Figure 7 Any first directed line segment Figure 7 Taking line segment 3 as an example, any second directed line segment is... Figure 7 Taking line segment 4 as an example, and considering that line segment 3 includes starting point 3 and line segment 4 includes starting point 4; after determining that line segment 3 and line segment 4 coincide, and that starting point 3 falls on line segment 4, then starting point 3 is determined as the intersection of the first bounding box and the second bounding box; if starting point 4 is determined to be on line segment 3, then starting point 4 is determined as the intersection of the first bounding box and the second bounding box. Specifically, the method for determining whether starting point 3 falls on line segment 4 is similar to the process for determining whether intersection point 1 falls on line segment 1, and will not be repeated here.

[0149] There are multiple ways to determine whether any first directed line segment coincides with any second directed line segment, including but not limited to the following:

[0150] Method 1: Any first directed line segment includes a first starting point and a first ending point, and any second directed line segment includes a second starting point and a second ending point; if the distance between the first starting point or the first ending point and any second directed line segment is less than a fourth threshold, then any first directed line segment and any second directed line segment are determined to coincide.

[0151] For example, please see Figure 7 Any first directed line segment Figure 7 Taking line segment 3 as an example, any second directed line segment is... Figure 7 Taking line segment 4 as an example, assuming the fourth threshold is 0.02cm, line segments 3 and 4 are parallel line segments, and line segment 3 includes the starting point 3 and the ending point 3, and line segment 4 includes the starting point 4 and the ending point 4; if the distance between the starting point 3 and / or the ending point 3 and line segment 4 is less than 0.02cm, then it is determined that line segment 3 and line segment 4 coincide.

[0152] Method 2: Any first directed line segment includes a first starting point and a first ending point, and any second directed line segment includes a second starting point and a second ending point; if the distance between the second starting point or the second ending point and any first directed line segment is less than a fifth threshold, then any first directed line segment is determined to coincide with any second directed line segment.

[0153] For example, please continue to see Figure 7 Any first directed line segment Figure 7Taking line segment 3 as an example, any second directed line segment is... Figure 7 Taking line segment 4 as an example, assuming the fifth threshold is 0.03cm, line segments 3 and 4 are parallel line segments, and line segment 3 includes the starting point 3 and the ending point 3, and line segment 4 includes the starting point 4 and the ending point 4; or, if the distance between the starting point 4 and / or the ending point 4 and line segment 3 is less than 0.03cm, then it is determined that line segment 3 and line segment 4 coincide.

[0154] Method 3: Any first directed line segment includes a first starting point and a first ending point, and any second directed line segment includes a second starting point and a second ending point; if the distance between any first directed line segment and any second directed line segment is less than a sixth threshold, then any first directed line segment and any second directed line segment are considered to coincide. The distance between any first directed line segment and any second directed line segment can be geometrically calculated using a first distance between the first starting point and the second ending point, and a second distance between the first ending point and the second starting point.

[0155] For example, please continue to see Figure 7 Any first directed line segment Figure 7 Taking line segment 3 as an example, any second directed line segment is... Figure 7 Taking line segment 4 as an example, assuming the sixth threshold is 0.01cm, line segments 3 and 4 are parallel line segments, and line segment 3 includes the starting point 3 and the ending point 3, and line segment 4 includes the starting point 4 and the ending point 4; the distance between the starting point 3 and the ending point 4 is distance 1, and the distance between the ending point 3 and the starting point 4 is distance 2. The distance between line segment 3 and line segment 4 determined by distance 1 and distance 2 is 0.005, which is less than the sixth threshold, so it is determined that line segment 3 and line segment 4 coincide.

[0156] In methods 1 to 3, considering that the coordinate values ​​of each point on any first directed line segment and any second directed line segment are irrational numbers, the overlap between any first directed line segment and any second directed line segment is determined by approximation, and the overlapping line segments are flexibly determined, thereby making the intersection points determined based on the overlapping line segments more accurate.

[0157] It should be understood that the specific values ​​of the first, second, third, fourth, fifth, and sixth thresholds mentioned above may be the same or different, and the embodiments of this application do not impose specific limitations. The specific implementation methods for determining the intersection points of parallel line segments and intersecting line segments described above can be implemented individually or in combination with each other.

[0158] Step 2: Based on at least two intersection points, split the first bounding box into at least two first directed line segments, and split the second bounding box into at least two second directed line segments. Each first directed line segment and each second directed line segment includes a start point and an end point. Hereinafter, "directed line segment" will be simply referred to as "line segment," and all line segments involved in this embodiment refer to "directed line segments."

[0159] For example, such as Figure 8 As shown in (a), the first bounding box and the second bounding box have 4 intersection points. Based on these 4 intersection points, the first bounding box can be divided into line segment 1, line segment 2, line segment 3, and line segment 4, and the second bounding box can be divided into line segment 5, line segment 6, line segment 7, and line segment 8.

[0160] Step 3: Based on the first positional relationship (e.g., intersecting, tangent, etc.), determine the first directed line segment located outside the second bounding box and the second directed line segment located inside the first bounding box.

[0161] Furthermore, a first directed line segment located within the second bounding box and a second directed line segment located outside the first bounding box can be determined based on the first positional relationship.

[0162] Optionally, at least two second directed line segments may contain a second directed line segment located on the first bounding box. Such a second directed line segment is determined to be a second directed line segment located within the first bounding box. Correspondingly, at least two first directed line segments may contain a first directed line segment located on the second bounding box. Such a first directed line segment is determined to be a first directed line segment located within the second bounding box. This allows for flexible determination of the relationship between line segments and bounding boxes.

[0163] For example, such as Figure 8 As shown in (a), the first bounding box and the second bounding box intersect. The first bounding box includes line segments 1, 2, 3, and 4, and the second bounding box includes line segments 5, 6, 7, and 8. Line segments 1 and 3 are located inside the second bounding box, so they are considered line segments within the second bounding box. Line segment 4 is located outside the second bounding box, so it is considered a line segment outside the second bounding box. Line segments 5 and 7 are located inside the first bounding box, so they are considered line segments within the first bounding box. Line segment 8 is located outside the first bounding box, so it is considered a line segment outside the first bounding box. Line segment 2 is on the second bounding box, so it is considered a line segment within the second bounding box. Line segment 6 is on the first bounding box, so it is considered a line segment within the first bounding box.

[0164] Step 4: Determine the updated first bounding box based on the first directed line segment located outside the second bounding box and the second directed line segment located inside the first bounding box.

[0165] In one possible implementation, a first directed line segment located outside the second bounding box and a second directed line segment located inside the first bounding box are connected to obtain an updated first bounding box. This achieves automatic annotation of the first bounding box, thereby improving the annotation efficiency of the first image.

[0166] Furthermore, the first directed line segment located within the second bounding box can be deleted, while all second directed line segments within the second bounding box remain unchanged.

[0167] Combination Figure 8 The example shown is as follows: Figure 8 As shown in (a), line segments 1, 2, and 3 in the first bounding box are located inside the second bounding box, and line segment 4 is located outside the second bounding box; line segments 5, 6, and 7 in the second bounding box are located inside the first bounding box, and line segment 8 is located outside the first bounding box; by deleting line segments 1, 2, and 3, and connecting line segments 5, 6, 7, and 4, the following can be obtained: Figure 8 The updated first bounding box is shown in (b), while the second bounding box remains unchanged.

[0168] S403: Generate a second image based on the first image, the second bounding box, and the updated first bounding box.

[0169] It should be understood that the second image contains boundary markers, which can be used to indicate that the updated first bounding box and the second bounding box have the same first target directed line segment, that is, the boundary line segment common to the first bounding box and the second bounding box (e.g., Figure 8 The target line segment 1 is shown in (b).

[0170] S404: Input the second image into the recognition model for training to obtain the trained recognition model.

[0171] For example, several second images are used as input to the recognition model, and the boundary markers corresponding to the second images are used as output to train the recognition model. The trained recognition model can then be used to automatically identify the boundaries between different semantic objects in an image. For instance, after inputting a real road environment image to be recognized into the recognition model, the model can output the common boundary formed between vehicles and pedestrians in the environment image due to the existence of a first positional relationship, without the need for manual annotation.

[0172] As can be seen from the above description, the embodiments of this application can automatically annotate the boundary information between different semantic objects with fully annotated bounding boxes, effectively improving the annotation efficiency of the image to be annotated, achieving the effect of obtaining annotated sample data at low cost and high efficiency, thereby optimizing the accuracy and effectiveness of the recognition model.

[0173] It should be understood that the number of semantic objects in the first image is not specifically limited in the embodiments of this application. The above description of the first image including the first semantic object and the second semantic object is merely an example and not a limitation.

[0174] The following describes a case where the first image contains multiple semantic objects, using specific examples.

[0175] It should be noted that when the first image includes multiple semantic objects, after updating the first bounding box based on the positional relationship between the first and second bounding boxes, the positional relationship between the updated first bounding box and the bounding boxes of other semantic objects in the first image may change. Therefore, it is necessary to update the bounding box of the first semantic object again according to the actual situation. The process of updating the bounding box of the first semantic object will differ depending on the positional relationship between the first semantic object and other semantic objects.

[0176] In one possible embodiment, the first image includes a third semantic object in addition to the first and second semantic objects. Before the first bounding box of the first semantic object is updated, the first bounding box contains the third bounding box of the third semantic object. After the updated first bounding box is internally tangent to the third bounding box, the bounding box of the first semantic object is updated again, resulting in a second-updated first bounding box. Based on the first image, the second bounding box, the third bounding box, and the second-updated first bounding box, a fourth image is generated. Based on the fourth image, the trained recognition model is retrained to obtain a new recognition model. The third bounding box and the second-updated first bounding box share the same directed line segment representing the second target. This improves the boundary labeling accuracy of semantic objects with multiple positional relationships, effectively enhancing the accuracy of labeled samples and thus optimizing the recognition model.

[0177] For example, such as Figure 9 As shown in (a), the first semantic object is object A, the second semantic object is object B, and the third semantic object is object C. The bounding box of object A is inscribed in the bounding box of object B, and the bounding box of object B is circumscribed in the bounding box of object C.

[0178] After updating the bounding box of object A, the updated bounding box of object A is obtained as follows: Figure 9 The dashed box shown in (b) indicates that the updated bounding box of object A and the bounding box of object B have a second target directed line segment (i.e., Figure 9 (The bolded line segment shown in (b)); by Figure 9 As shown in (b), the positional relationship between the bounding box of object A and the bounding box of object C after the first update becomes inscribed, therefore the bounding box of object A needs to be updated again; after the second update of the bounding box of object A, we can obtain the following... Figure 9 The bounding box of object A after the second update shown in (c) has the same third target directed line segment as the bounding box of object C. Figure 9(The bolded line segment in the bounding box of object C in (c)). Furthermore, a fourth image can be generated based on the second-updated bounding box of object A, the bounding box of object B, and the corresponding bounding box of object C. Based on the fourth image, the trained recognition model can be trained again to obtain a new recognition model.

[0179] In one possible embodiment, when the first positional relationship between the first bounding box of the first semantic object and the second bounding box of the second semantic object is intersecting, there is a special case where the first bounding box of the first semantic object after one update obtained in S402 is truncated by the second bounding box. The updated first bounding box is divided into two bounding boxes, for example, a first sub-bounding box and a second sub-bounding box, wherein both the first sub-bounding box and the second sub-bounding box share a common boundary line segment with the second bounding box. Here, "truncated" can be understood as the first bounding box being occluded by the second bounding box. In this way, the first bounding box can be automatically updated through the first positional relationship, realizing automatic annotation of the first bounding box in the occlusion scenario, thereby improving the annotation efficiency.

[0180] For example, please see Figure 10 ,exist Figure 10 In (a), the first bounding box is taken as bounding box 1, and the second bounding box is taken as bounding box 2. Bounding box 1 and bounding box 2 intersect, and bounding box 1 is truncated by bounding box 2. Bounding box 1 is updated to obtain the following: Figure 10 In Figure (b), bounding boxes 3 and 4 are shown (i.e., the updated bounding box 1 is divided into bounding boxes 3 and 4), where bounding box 3 and bounding box 2 share a common boundary segment 1, and bounding box 4 and bounding box 2 share a common boundary segment 2.

[0181] Furthermore, if the first image also includes a third semantic object, the first bounding box may be truncated not only by the second semantic bounding box but also by the third bounding box of the third semantic object. After the first bounding box is updated once, a first sub-bounding box and a second sub-bounding box are obtained. If the first sub-bounding box and / or the second sub-bounding box are truncated by the third bounding box, the bounding box of the first semantic object is updated again to obtain at least three sub-bounding boxes.

[0182] For example, please continue to see Figure 10 In (c), the third semantic object corresponds to bounding box 5. Bounding box 4, obtained after the first bounding box is updated once, is truncated by bounding box 5. The first bounding box is updated a second time. The first bounding box after the second update includes bounding box 3, bounding box 6, and bounding box 7. Bounding box 6 and bounding box 5 have the same boundary line segment 3 (i.e., the bold line segment in bounding box 5), and bounding box 7 and bounding box 5 have the same boundary line segment 4 (i.e., the bold line segment in bounding box 5).

[0183] It should be understood that in practical application scenarios, the first bounding box may be truncated by the bounding boxes of multiple semantic objects. The above is merely an example of the first bounding box being truncated by the bounding boxes of one or two semantic objects, and is not intended to limit the embodiments of this application. In a specific example, the first semantic object can be a vehicle, and the second and third semantic objects can be tree branches, meaning that the above technical solution can be applied to application scenarios where the vehicle is obscured by tree branches.

[0184] In this embodiment, the bounding boxes between different semantic objects can be automatically labeled, effectively solving the problem that common boundaries between different semantic objects due to occlusion need to be drawn repeatedly, thereby effectively improving the labeling efficiency in the first image.

[0185] Please see Figure 11 To automatically annotate blank areas in images to be annotated, this application provides another method for training a recognition model, which specifically includes:

[0186] S1101: Obtain the first image. The first image includes at least two semantic objects, and the at least two semantic objects include a first semantic object and a second semantic object.

[0187] For a detailed description of S1101, please refer to the description of S401 above, which will not be repeated here.

[0188] S1102: Determine the bounding box of the blank area in the first image based on the first bounding box of the first semantic object and the second bounding box of the second semantic object.

[0189] Specifically, the bounding box of the blank area determined in S1102 shares the same third target directed line segment as the first bounding box and the same fourth target directed line segment as the second bounding box. Here, the "third target directed line segment" refers to the common boundary line segment between the blank area's bounding box and the first bounding box, such as the common boundary line segment between the road surface and the vehicle's respective bounding boxes; the "fourth target directed line segment" refers to the common boundary line segment between the blank area's bounding box and the second bounding box, such as the common boundary line segment between the road surface's bounding box and the green belt.

[0190] There are multiple implementation methods for determining the boundary of the blank area based on the first boundary box and the second boundary box.

[0191] In one possible implementation, the first bounding box is divided into at least one first directed line segment, and the second bounding box is divided into at least one second directed line segment. A portion of the first directed line segment in the first bounding box and a portion of the second directed line segment in the second bounding box enclose a blank area. Then, by connecting the portion of the first directed line segment and the portion of the second directed line segment, the bounding box of the blank area can be obtained.

[0192] For example, please see Figure 12A The first bounding box is taken as the bounding box of object 1, and the second bounding box is taken as the bounding box of object 2. Partial line segments within the bounding box of object 1 and partial line segments within the bounding box of object 2 (i.e....) Figure 12A The bolded line segments in the image enclose a blank area 1. By simply obtaining the information of these line segments and automatically connecting them, the bounding box of the blank area 1 can be obtained. Figure 12A The bounding box is composed of the bold line segments in the text.

[0193] In another possible implementation, the first bounding box is split into at least one first directed line segment, and the second bounding box is split into at least one second directed line segment. A portion of the first directed line segment in the first bounding box, a portion of the second directed line segment in the second bounding box, and the boundary of the first image form a blank area. Then, the boundary information of the first image is obtained, and the first directed line segment, the second directed line segment, and the boundary of the first image are connected to obtain the bounding box of the blank area.

[0194] For example, please see Figure 12B The first bounding box is taken as the bounding box of object 1, and the second bounding box is taken as the bounding box of object 2. The bounding boxes of object 1 and object 2 each contain some line segments, and the bounding boxes of the first image also contain some line segments (i.e.,...) Figure 12A The bolded line segments in the image enclose a blank area 2. By obtaining the specific information of these line segments and automatically connecting them, the bounding box of the blank area 2 can be obtained. Figure 12B The bounding box is composed of the bold line segments in the text.

[0195] Specifically, the process of segmenting the first bounding box can be performed by dividing the first bounding box into segments based on the intersection points between the first bounding box and the boundary segments of the first image. Similarly, the process of segmenting the second bounding box can be performed by dividing the second bounding box into segments based on the intersection points between the second bounding box and the boundary segments of the first image. The specific implementation method for determining the intersection points is similar to the implementation method for determining the intersection points of the first and second bounding boxes described above, and will not be repeated here.

[0196] S1103: Generate a second image based on the first image, the second bounding box, the first bounding box, and the bounding box of the blank area;

[0197] It should be understood that the second image contains boundary markers, which are used to indicate the bounding boxes of blank areas (e.g., Figure 12A (The bounding box of the blank area 1 shown).

[0198] S1104: Input the second image into the recognition model for training to obtain the trained recognition model.

[0199] In one possible implementation, after the recognition model is obtained through the second image optimization training, a new image is input into the recognition model, which can output the bounding box information of the blank area in the image, without the need for manual annotation of the boundary of the blank area.

[0200] In this embodiment, the bounding boxes of the semantic objects already marked in the first image (e.g., the first bounding box and the second bounding box) can be used to determine the bounding boxes of the blank areas, effectively improving the annotation efficiency of the blank areas in the first image, thereby effectively improving the efficiency of obtaining sample data and achieving the effect of optimizing the accuracy of the recognition model. Combined with autonomous driving scenarios, this recognition model can automatically identify the blank areas on the road surface, facilitating vehicle route planning and enabling better autonomous driving.

[0201] Please see Figure 13 To automatically annotate isolated regions in an image to be annotated, this application provides another method for training a recognition model, which specifically includes:

[0202] S1301: Obtain a first image. The first image includes at least one semantic object, and the at least one semantic object includes the first semantic object.

[0203] For a detailed description of S1301, please refer to the description of S401 above, which will not be repeated here.

[0204] It should be understood that the complete bounding box of each semantic object in the first image has been marked, such as the first bounding box of the first semantic object.

[0205] S1302: Determine the bounding box of the isolated region in the first image based on the first bounding box of the first semantic object and the boundary of the first image.

[0206] Specifically, the bounding box of the isolated region includes at least two bounding boxes, and any one of the at least two bounding boxes has the same third target directed line segment as the first bounding box, that is, any one of the bounding boxes has the same boundary line segment as the first bounding box.

[0207] For example, such as Figure 14AAs shown, taking the bounding box of object 1 as an example, the first image is isolated by the bounding box of object 1, resulting in isolated region 1, isolated region 2, and isolated region 3. The bounding box of isolated region 1 has the same boundary line segment 1 (i.e., the thickest line segment in the bounding box of object 1) as the bounding box of object 1. The bounding box of isolated region 2 has the same boundary line segment 2 (i.e., the thinnest line segment in the bounding box of object 1) as the bounding box of object 1. The bounding box of isolated region 3 has the same boundary line segment 3 (i.e., the thicker line segment in the bounding box of object 1) as the bounding box of object 1.

[0208] In one possible implementation, the bounding box of isolated region 1 can be obtained by automatically connecting the common boundary segment 1 between the bounding box of isolated region 1 and the bounding box of object 1, as well as the boundary segment on the right side of the first image. The method for determining the bounding boxes of isolated regions 2 and 3 is similar and will not be described again.

[0209] In practical applications, the isolated region can also be obtained by separating the regions using the bounding boxes of multiple semantic objects in the first image (e.g., the first bounding box of the first semantic object and the second bounding box of the second semantic object). For an example, please refer to [link to example]. Figure 14B The first bounding box is taken as the bounding box of object 1, and the second bounding box is taken as the bounding box of object 2. The first image is isolated by the bounding boxes of object 1 and object 2, resulting in isolated region 1, isolated region 2, isolated region 3, and isolated region 4.

[0210] S1303: Generate a second image based on the first image, the first bounding box, and the bounding box of the isolated region.

[0211] It should be understood that the second image contains boundary markers, which are used to indicate the bounding boxes of the isolated areas (e.g., Figure 14B The bounding boxes of isolation region 1 and isolation region 2 are shown.

[0212] S1304: Input the second image into the recognition model for training to obtain the trained recognition model.

[0213] In one possible implementation, after the recognition model is obtained through the second image optimization training, a new image is input into the recognition model, and the recognition model can output the bounding box information of the isolated region in the image, without the need for manual annotation of the boundary of the isolated region.

[0214] In this embodiment of the application, the bounding boxes of semantic objects already marked in the first image (e.g., Figure 14BBy identifying the bounding boxes of object 1 and object 2, the bounding boxes of the blank areas can be determined, effectively improving the efficiency of labeling isolated areas in the first image, thereby improving the efficiency of obtaining sample data and optimizing the accuracy of the recognition model. In the context of autonomous driving, this recognition model can automatically identify isolated areas on the road surface formed by green belts and vehicles, facilitating vehicle route planning and enabling functions such as autonomous driving and assisted driving.

[0215] In this embodiment, by using the bounding boxes of the semantic objects already labeled in the first image (e.g., the first bounding box) and the boundary information of the first image, the bounding boxes corresponding to the two isolated regions formed by the first bounding box in the first image can be automatically determined, which effectively improves the labeling efficiency of the isolated regions in the first image, thereby effectively improving the efficiency of obtaining sample data and achieving the effect of optimizing the accuracy of the recognition model.

[0216] In practical applications, the training methods for the three recognition models described above can also be applied to the field of image processing. For example, they can be used in image processing software.

[0217] In one implementation, in Figure 4 The training method of the recognition model shown is applied to image processing software. This software can automatically update the layers containing the first and second semantic objects in the image being processed, which are subject to occlusion. For example, if the layer containing a person is occluded by the layer containing a tree, the layer containing the person is automatically updated to create a boundary marker between the person and the tree due to the occlusion. This makes the image more realistic.

[0218] In another implementation, in Figure 11 The training method of the recognition model shown is applied to image processing software. This software can automatically extract the blank areas formed by the bounding boxes of different semantic objects in the image to be processed and the bounding boxes corresponding to the boundaries of the image to be processed. For example, it can determine the bounding boxes of image regions other than people in an image. In this way, the efficiency of extracting layers can be effectively improved, thereby effectively saving labor costs.

[0219] In another implementation, in Figure 13 The training method of the recognition model shown is applied to image processing software. This software can be used to automatically extract the layer containing the isolated region formed by the bounding box of a specifically defined object in the image to be processed, such as determining the bounding box of the image region isolated by a green belt in an image. This effectively improves the efficiency of extracting layers, thereby significantly saving labor costs.

[0220] The above embodiments can be combined with each other to achieve different technical effects.

[0221] The apparatus used to implement the above method in the embodiments of this application is described below with reference to the accompanying drawings. Therefore, the content above can be used in subsequent embodiments, and repeated content will not be described again.

[0222] Figure 15 This diagram illustrates a possible structure of the model training apparatus involved in the above embodiments of this application. The apparatus 1500 can be used to implement the above-described... Figure 4 The method in the illustrated embodiment.

[0223] For example, device 1500 may include:

[0224] The acquisition module 1501 is used to acquire a first image, the first image including at least two semantic objects, the at least two semantic objects including a first semantic object and a second semantic object;

[0225] The processing module 1502 is used to update the bounding box of the first semantic object according to the first positional relationship between the first bounding box of the first semantic object and the second bounding box of the second semantic object, to obtain the updated first bounding box of the first semantic object; wherein, the first positional relationship includes any one of external tangency, internal tangency, and intersection, and the updated first bounding box and the second bounding box have the same first target directed line segment; generate a second image based on the first image, the second bounding box, and the updated first bounding box; input the second image into the recognition model for training to obtain the trained recognition model.

[0226] In one possible design, before updating the bounding box of the first semantic object, the acquisition module 1501 can also be used to receive a first operation; based on the first operation, determine that the first bounding box is the first layer, the second bounding box is the second layer, and the first target directed line segment is located on the second layer.

[0227] In one possible design, when the acquisition module 1501 is used to acquire the first image, it is specifically used to: acquire the bounding box of at least one semantic object, randomly combine the bounding boxes of the at least one semantic object to obtain the first image; and / or acquire a third image, process the third image to obtain the first image.

[0228] In one possible design, when processing module 1502 updates the bounding box of the first semantic object based on the first positional relationship between the first bounding box of the first semantic object and the second bounding box of the second semantic object, it specifically performs the following steps: determining at least two intersection points of the first and second bounding boxes; dividing the first bounding box into at least two first directed line segments and the second bounding box into at least two second directed line segments based on the at least two intersection points; determining the first directed line segments located outside the second bounding box and the second directed line segments located inside the first bounding box based on the first positional relationship; and determining the updated first bounding box based on the first directed line segments located outside the second bounding box and the second directed line segments located inside the first bounding box. Each first directed line segment and each second directed line segment includes a start point and an end point.

[0229] In one possible design, when the processing module 1502 determines the updated first bounding box based on the first directed line segment located outside the second bounding box and the second directed line segment located inside the first bounding box, it specifically performs the following: connecting the first directed line segment located outside the second bounding box and the second directed line segment located inside the first bounding box to obtain the updated first bounding box.

[0230] In one possible design, when the processing module 1502 determines, based on the first positional relationship, that there is a second directed line segment located on the first bounding box among at least two second directed line segments, then the second directed line segment is determined to be a second directed line segment located within the first bounding box; when the processing module 1502 determines, based on the first positional relationship, that there is a first directed line segment located on the second bounding box among at least two first directed line segments, then the first directed line segment is determined to be a first directed line segment located within the second bounding box.

[0231] In one possible design, when the processing module 1502 is used to determine at least two intersection points of the first bounding box and the second bounding box, it is specifically used to: determine that any first directed line segment is not parallel to any second directed line segment and that a first intersection point exists; and determine that the first intersection point is on any first directed line segment and any second directed line segment; if the first intersection point is not the endpoint of any first directed line segment and is not the endpoint of any second directed line segment, then the first intersection point is determined as any one of the at least two intersection points.

[0232] In one possible design, when the processing module 1502 determines that the first intersection point is on any first directed line segment and any second directed line segment, it specifically performs the following steps: determining that the distance between the first intersection point and any first directed line segment is less than a first threshold, and determining that the first intersection point is on any first directed line segment; and / or, determining that the distance between the first intersection point and any second directed line segment is less than a second threshold, and determining that the first intersection point is on any second directed line segment.

[0233] In one possible design, when the processing module 1502 is used to determine at least two intersection points of the first bounding box and the second bounding box, it is specifically used to: determine that there is a second intersection point between any first directed line segment and any second directed line segment, and the distance between the second intersection point and the first intersection point is less than a third threshold, then the second intersection point and the first intersection point are regarded as the same intersection point.

[0234] In one possible design, determining at least two intersection points of the first and second bounding boxes may include: determining that any first directed line segment is parallel to any second directed line segment; any first directed line segment includes a first starting point and a first ending point, and any second directed line segment includes a second starting point and a second ending point; determining that any first directed line segment coincides with any second directed line segment if the distance between the first starting point or the first ending point and any second directed line segment is less than a fourth threshold; or, determining that any first directed line segment coincides with any second directed line segment if the distance between the second starting point or the second ending point and any first directed line segment is less than a fifth threshold; and determining the first starting point and / or the second starting point as any of the at least two intersection points.

[0235] In one possible design, when the second starting point is on any first directed line segment, the processing module 1502 can determine the second starting point as any intersection of the first bounding box and the second bounding box; or, when the first starting point is on any second directed line segment, the processing module 1502 can determine the first starting point as any intersection of the first bounding box and the second bounding box.

[0236] In one possible design, at least two semantic objects also include a third semantic object; after obtaining the updated first bounding box of the first semantic object, the processing module 1502 is further configured to: determine that the third bounding box of the third semantic object is located within the first bounding box; and when it is determined that the third bounding box of the third semantic object is internally tangent to the updated first bounding box, update the bounding box of the first semantic object again to obtain the first bounding box of the first semantic object after a second update; wherein the third bounding box and the first bounding box after a second update have the same second target directed line segment; generate a fourth image based on the first image, the second bounding box, the third bounding box, and the first bounding box after a second update; and retrain the trained recognition model based on the fourth image to obtain a new recognition model.

[0237] In one possible design, when the first positional relationship is intersecting, the updated first bounding box includes a first sub-bounding box and a second sub-bounding box. That is, the first bounding box is truncated by the second bounding box, where "truncated" can be understood as the first bounding box being occluded by the second bounding box. Both the first and second sub-bounding boxes share common boundary segments with the second bounding box.

[0238] In one possible design, at least two semantic objects also include a third semantic object; after obtaining the updated first bounding box of the first semantic object, the processing module 1502 is further configured to: determine that the first sub-bounding box and / or the second sub-bounding box intersect with the third bounding box of the third semantic object, update the bounding box of the first semantic object again, and obtain the second updated first bounding box; wherein the second updated first bounding box includes three sub-bounding boxes.

[0239] Figure 16 This diagram illustrates a possible structure of the model training apparatus involved in the above embodiments of this application. The apparatus 1600 can be used to implement the above-described... Figure 11 The method in the illustrated embodiment.

[0240] For example, device 1600 may include:

[0241] The acquisition module 1601 is used to acquire a first image, the first image including at least two semantic objects, the at least two semantic objects including a first semantic object and a second semantic object;

[0242] The processing module 1602 is used to determine the bounding box of the blank area in the first image based on the first bounding box of the first semantic object and the second bounding box of the second semantic object; wherein the bounding box of the blank area has the same third target directed line segment as the first bounding box and the same fourth target directed line segment as the second bounding box; generate a second image based on the first image, the second bounding box, the first bounding box, and the bounding box of the blank area; input the second image into the recognition model for training to obtain the trained recognition model.

[0243] Figure 17 This diagram illustrates a possible structure of the model training apparatus involved in the above embodiments of this application. The apparatus 1700 can be used to implement the above-described... Figure 13 The method in the illustrated embodiment.

[0244] For example, device 1700 may include:

[0245] The acquisition module 1701 is used to acquire a first image, the first image including at least two semantic objects, the at least two semantic objects including a first semantic object and a second semantic object;

[0246] Processing module 1702 is configured to determine the bounding box of an isolated region in the first image based on the first bounding box of the first semantic object and the boundary of the first image; wherein the isolated region is a region that does not intersect with the bounding box of at least one semantic object; the bounding box of the isolated region includes at least two bounding boxes, any one of the at least two bounding boxes having the same third target directed line segment as the first bounding box, and / or having the same fourth target directed line segment as the second bounding box; generate a second image based on the first image, the first bounding box, and the bounding box of the isolated region; input the second image into the recognition model for training to obtain the trained recognition model.

[0247] Based on the same technical concept, this application also provides an electronic device 1800 for implementing... Figure 4 , Figure 11 , Figure 13 The method in the illustrated embodiment.

[0248] like Figure 18 As shown, the electronic device 1800 may include a processor 1801 for executing programs or instructions stored in a memory 1802. When the programs or instructions stored in the memory 1802 are executed, the processor is used to execute... Figure 4 , Figure 11 , Figure 13 The method in the illustrated embodiment.

[0249] Optionally, the electronic device 1800 may also include a communication interface 1803. (See appendix) Figure 18 The dashed line indicates that the communication interface 1803 is optional for electronic device 1800.

[0250] The number of processors 1801, memory 1802 and communication interfaces 1803 does not constitute a limitation on the embodiments of this application. In specific implementation, they can be configured arbitrarily according to business needs.

[0251] Optionally, the memory 1802 is located outside the electronic device 1800.

[0252] Optionally, the electronic device 1800 includes the memory 1802, which is connected to the at least one processor 1801. The memory 1802 stores instructions that can be executed by the at least one processor 1801. (Appendix) Figure 18 The dashed line indicates that memory 1802 is optional for electronic device 1800.

[0253] The processor 1801 and the memory 1802 can be coupled through an interface circuit or integrated together; no restriction is imposed here.

[0254] This application embodiment does not limit the specific connection medium between the processor 1801, memory 1802, and communication interface 1803. This application embodiment... Figure 18 The processor 1801, memory 1802, and communication interface 1803 are connected via a bus 1804. Figure 18 The connections between other components are shown in bold and are for illustrative purposes only, not as limiting information. The bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, Figure 18 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0255] It should be understood that the processor mentioned in the embodiments of this application can be implemented in hardware or software. When implemented in hardware, the processor can be a logic circuit, integrated circuit, etc. When implemented in software, the processor can be a general-purpose processor, implemented by reading software code stored in memory.

[0256] For example, the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor.

[0257] It should be understood that the memory mentioned in the embodiments of this application can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate Synchronous DRAM (DDR SDRAM), Enhanced Synchronous DRAM (ESDRAM), Synchlink DRAM (SLDRAM), and Direct RAM (DR RAM).

[0258] It should be noted that when the processor is a general-purpose processor, DSP, ASIC, FPGA, or other programmable logic device, discrete gate or transistor logic device, or discrete hardware component, the memory (storage module) can be integrated into the processor.

[0259] It should be noted that the memories described herein are intended to include, but are not limited to, these and any other suitable types of memories.

[0260] This application embodiment also provides a vehicle, which may include a processor for performing the above-described actions. Figure 4 , Figure 11 , Figure 13 The training method for the recognition model described in the illustrated embodiment.

[0261] This application embodiment also provides a server, which includes a processor, the processor being used to perform the above-described... Figure 4 , Figure 11 , Figure 13 The training method for the recognition model described in the illustrated embodiment.

[0262] In one possible design, the server is a single server or a server cluster consisting of multiple sub-servers. When the server is a server cluster consisting of multiple sub-servers, the multiple sub-servers jointly execute the above-mentioned... Figure 4 , Figure 11 , Figure 13 The training method for the recognition model described in the illustrated embodiment.

[0263] This application also provides a chip system including at least one processor, wherein when program instructions are executed in the at least one processor, the above-mentioned... Figure 4 , Figure 11 , Figure 13 The training method for the recognition model described in the illustrated embodiment is thus implemented.

[0264] In one possible design, the chip system also includes a communication interface for inputting or outputting information.

[0265] In one possible design, the chip system also includes a memory coupled to the processor via a communication interface for storing the aforementioned instructions, so that the processor can read the instructions stored in the memory via the communication interface.

[0266] In one possible design, the processor described above can be a processing circuit, which is not limited in this application.

[0267] This application also provides a computer program product including instructions, which, when run on the aforementioned device, performs the above-described actions. Figure 4 , Figure 11 , Figure 13 The training method for the recognition model described in the illustrated embodiment.

[0268] This application provides a computer-readable storage medium storing a computer program that, when executed, performs the above-described functionality. Figure 4 , Figure 11 , Figure 13 The training method for the recognition model described in the illustrated embodiment.

[0269] The above embodiments can be combined with each other to achieve different technical effects.

[0270] Through the above description of the embodiments, those skilled in the art can clearly understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0271] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0272] The units described as separate components may or may not be physically separate. A component shown as a unit can be one or more physical units; that is, it can be located in one place or distributed in multiple different locations. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0273] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0274] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, essentially, or the parts that contribute to the prior art, or all or part of the technical solutions, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, ROM, RAM, magnetic disks, or optical disks.

[0275] The above description is merely a specific implementation of the embodiments of this application, but the protection scope of the embodiments of this application is not limited thereto. Any changes or substitutions within the technical scope disclosed in this application should be covered within the protection scope of the embodiments of this application. Therefore, the protection scope of the embodiments of this application should be determined by the protection scope of the claims.

Claims

1. A method for training a recognition model, characterized in that, include: Acquire a first image, the first image including at least two semantic objects, the at least two semantic objects including a first semantic object and a second semantic object; Based on the first positional relationship between the first bounding box of the first semantic object and the second bounding box of the second semantic object, the bounding box of the first semantic object is updated to obtain the updated first bounding box of the first semantic object; wherein, the first positional relationship includes one or more of external tangency, internal tangency, and intersection; the updated first bounding box and the second bounding box have the same first target directed line segment, and the first target directed line segment is the common boundary line segment of the updated first bounding box and the second bounding box; Based on the first image, the second bounding box, and the updated first bounding box, a second image is generated. The second image includes boundary markers, which are used to indicate the directed line segment of the first target. The second image is input into the recognition model for training to obtain the trained recognition model. During training, the boundary markers are used as supervision data for the recognition model, and the trained recognition model is used to identify the boundaries between different semantic objects in the image.

2. The method as described in claim 1, characterized in that, Before updating the bounding box of the first semantic object, the following steps are also included: Receive the first operation; Based on the first operation, the first bounding box is determined to be the first layer, the second bounding box is determined to be the second layer, and the first target directed line segment is located on the second layer.

3. The method as described in claim 1 or 2, characterized in that, The acquisition of the first image includes: Obtain the bounding boxes of at least one semantic object, and randomly combine the bounding boxes of the at least one semantic object to obtain the first image; and / or, A third image is acquired, and the third image is processed to obtain the first image.

4. The method according to any one of claims 1-3, characterized in that, The step of updating the bounding box of the first semantic object based on the first positional relationship between the first bounding box of the first semantic object and the second bounding box of the second semantic object includes: Determine at least two intersection points of the first bounding box and the second bounding box; based on the at least two intersection points, split the first bounding box into at least two first directed line segments, and split the second bounding box into at least two second directed line segments; each first directed line segment and each second directed line segment includes a start point and an end point; Based on the first positional relationship, a first directed line segment located outside the second bounding box and a second directed line segment located inside the first bounding box are determined; The updated first bounding box is determined based on the first directed line segment located outside the second bounding box and the second directed line segment located inside the first bounding box.

5. The method as described in claim 4, characterized in that, The updated first bounding box is determined based on the first directed line segment located outside the second bounding box and the second directed line segment located inside the first bounding box, including: The first directed line segment located outside the second bounding box and the second directed line segment located inside the first bounding box are connected to obtain the updated first bounding box.

6. The method as described in claim 4 or 5, characterized in that, Determining the second directed line segment located within the first bounding box based on the first positional relationship includes: When it is determined based on the first positional relationship that there is a second directed line segment located on the first bounding box among the at least two second directed line segments, then it is determined that the second directed line segment is a second directed line segment located inside the first bounding box.

7. The method according to any one of claims 4-6, characterized in that, Determining at least two intersection points of the first bounding box and the second bounding box includes: It is determined that any of the first directed line segments and any of the second directed line segments are not parallel, and there is a first intersection point; Determine that the first intersection point lies on either the first directed line segment or either the second directed line segment; If the first intersection point is not the endpoint of any of the first directed line segments, and is not the endpoint of any of the second directed line segments, then the first intersection point is determined as any one of the at least two intersection points.

8. The method as described in claim 7, characterized in that, Determining that the first intersection point lies on either the first directed line segment or the second directed line segment includes: Determine that the distance between the first intersection point and any of the first directed line segments is less than a first threshold, and determine that the first intersection point lies on any of the first directed line segments; and / or, If the distance between the first intersection point and any of the second directed line segments is less than a second threshold, then the first intersection point is determined to be on any of the second directed line segments.

9. The method as described in claim 7 or 8, characterized in that, Determining at least two intersection points of the first bounding box and the second bounding box includes: Determine that any of the first directed line segments and any of the second directed line segments have a second intersection point; When the distance between the second intersection point and the first intersection point is less than the third threshold, the second intersection point and the first intersection point are considered as the same intersection point.

10. The method according to any one of claims 4-6, characterized in that, Determining at least two intersection points of the first bounding box and the second bounding box includes: It is determined that any first directed line segment is parallel to any second directed line segment; any first directed line segment includes a first starting point and a first ending point, and any second directed line segment includes a second starting point and a second ending point; If the distance between the first starting point or the first ending point and any second directed line segment is less than a fourth threshold, then any first directed line segment coincides with any second directed line segment; or, if the distance between the second starting point or the second ending point and any first directed line segment is less than a fifth threshold, then any first directed line segment coincides with any second directed line segment. The first starting point and / or the second starting point are determined as any one of the at least two intersection points.

11. The method as described in claim 10, characterized in that, The step of determining the first starting point and / or the second starting point as any one of the at least two intersection points includes: When the second starting point lies on any of the first directed line segments, the second starting point is determined as any of the intersection points; or, When the first starting point is on any of the second directed line segments, the first starting point is determined as any of the intersection points.

12. The method according to any one of claims 1-11, characterized in that, The at least two semantic objects also include a third semantic object; After obtaining the updated first bounding box of the first semantic object, the method further includes: Determine that the third bounding box of the third semantic object is located within the first bounding box; The third bounding box of the third semantic object is determined to be internally tangent to the updated first bounding box. The bounding box of the first semantic object is then updated again to obtain the first bounding box of the first semantic object after the second update. The third bounding box and the first bounding box after the second update have the same second target directed line segment. A fourth image is generated based on the first image, the second bounding box, the third bounding box, and the second-updated first bounding box. Based on the fourth image, the trained recognition model is trained again to obtain a new recognition model.

13. The method according to any one of claims 1-11, characterized in that, When the first positional relationship is intersecting, the updated first bounding box includes a first sub-bounding box and a second sub-bounding box.

14. The method as described in claim 13, characterized in that, The at least two semantic objects also include a third semantic object; After obtaining the updated first bounding box of the first semantic object, the method further includes: Determine that the first sub-boundary box and / or the second sub-boundary box intersect with the third bounding box of the third semantic object, and update the bounding box of the first semantic object again to obtain the first bounding box after the second update; wherein, the first bounding box after the second update includes three sub-boundary boxes.

15. A method for training a recognition model, characterized in that, include: Acquire a first image, the first image including at least two semantic objects, the at least two semantic objects including a first semantic object and a second semantic object; Based on the first bounding box of the first semantic object and the second bounding box of the second semantic object, the bounding box of the blank area in the first image is determined; wherein, the blank area is the image area corresponding to the semantic object set in the first image, the bounding box of the blank area has the same third target directed line segment as the first bounding box, the bounding box of the blank area has the same fourth target directed line segment as the second bounding box, the third target directed line segment is the boundary line segment common to the bounding box of the blank area and the first bounding box, and the fourth target directed line segment is the boundary line segment common to the bounding box of the blank area and the second bounding box; A second image is generated based on the first image, the second bounding box, the first bounding box, and the bounding box of the blank area. The second image includes a boundary marker, which is used to indicate the bounding box of the blank area. The second image is input into the recognition model for training to obtain the trained recognition model. During training, the boundary markers are used as supervision data for the recognition model, and the trained recognition model is used to identify the boundaries of blank areas in the image.

16. A method for training a recognition model, characterized in that, include: Obtain a first image, the first image including at least one semantic object, the at least one semantic object including the first semantic object; Based on the first bounding box of the first semantic object and the boundary of the first image, the bounding box of the isolated region in the first image is determined; wherein, the isolated region is a region that does not intersect with the bounding box of the at least one semantic object; the bounding box of the isolated region includes at least two bounding boxes, and any one of the at least two bounding boxes has the same third target directed line segment as the first bounding box, wherein the third target directed line segment is the boundary line segment shared by the any bounding box and the first bounding box; Based on the first image, the first bounding box, and the bounding box of the isolated region, a second image is generated. The second image includes a boundary marker, which is used to indicate the bounding box of the isolated region. The second image is input into the recognition model for training to obtain the trained recognition model. During training, the boundary markers are used as supervision data for the recognition model, and the trained recognition model is used to identify the boundaries of isolated regions in the image.

17. A training device for a recognition model, characterized in that, include: An acquisition module is used to acquire a first image, the first image including at least two semantic objects, the at least two semantic objects including a first semantic object and a second semantic object; The processing module is configured to update the bounding box of the first semantic object based on a first positional relationship between the first bounding box of the first semantic object and the second bounding box of the second semantic object, thereby obtaining an updated first bounding box of the first semantic object; wherein the first positional relationship includes any one of external tangency, internal tangency, and intersection, and the updated first bounding box and the second bounding box have the same first target directed line segment, which is a common boundary line segment of the updated first bounding box and the second bounding box; generate a second image based on the first image, the second bounding box, and the updated first bounding box, the second image including boundary markers, the boundary markers being used to indicate the first target directed line segment; input the second image into the recognition model for training, thereby obtaining a trained recognition model, wherein during training, the boundary markers are used as supervision data for the recognition model, and the trained recognition model is used to identify the boundaries between different semantic objects in the image.

18. A training device for a recognition model, characterized in that, include: An acquisition module is used to acquire a first image, the first image including at least two semantic objects, the at least two semantic objects including a first semantic object and a second semantic object; The processing module is configured to determine the bounding box of a blank area in the first image based on the first bounding box of the first semantic object and the second bounding box of the second semantic object; wherein the blank area is an image region corresponding to a defined semantic object in the first image, the bounding box of the blank area has the same third target directed line segment as the first bounding box, the bounding box of the blank area has the same fourth target directed line segment as the second bounding box, the third target directed line segment is a common boundary line segment of the bounding box of the blank area and the first bounding box, and the fourth target directed line segment is a common boundary line segment of the bounding box of the blank area and the second bounding box; generate a second image based on the first image, the second bounding box, the first bounding box, and the bounding box of the blank area, the second image including boundary markers, the boundary markers being used to indicate the bounding box of the blank area; input the second image into the recognition model for training to obtain a trained recognition model, wherein, during training, the boundary markers are used as supervision data for the recognition model, and the trained recognition model is used to recognize the boundaries of blank areas in the image.

19. A training device for a recognition model, characterized in that, include: An acquisition module is used to acquire a first image, the first image including at least one semantic object, the at least one semantic object including the first semantic object; A processing module is configured to determine the bounding box of an isolated region in the first image based on the first bounding box of the first semantic object and the boundary of the first image; wherein the isolated region is a region that does not intersect with the bounding box of the at least one semantic object; the bounding box of the isolated region includes at least two bounding boxes, any one of the at least two bounding boxes having the same third target directed line segment as the first bounding box, the third target directed line segment being the common boundary line segment of the any bounding box and the first bounding box; generate a second image based on the first image, the first bounding box, and the bounding box of the isolated region, the second image including boundary markers, the boundary markers being used to indicate the bounding box of the isolated region; input the second image into the recognition model for training to obtain a trained recognition model, wherein, during training, the boundary markers are used as supervision data for the recognition model, and the trained recognition model is used to recognize the boundary of the isolated region in the image.

20. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed, implements the method as described in any one of claims 1 to 16.

21. A chip, characterized in that, It includes a processor and a communication interface, the communication interface being used for inputting or outputting information; the processor being used for reading instructions to execute the method according to any one of claims 1 to 16.

22. A vehicle, characterized in that, Includes a processor for performing the method according to any one of claims 1 to 16.

23. A server, characterized in that, Includes a processor for performing the method according to any one of claims 1 to 16.

Images