A training method, an image processing method, and related devices

Abstract

The application provides a training method, an image processing method and related equipment. The method comprises: determining first boundary information representing the boundary of a first elliptical region based on recognition boundary information, the first elliptical region being a region with the same center and size as the maximum inscribed ellipse of a recognition rectangular region; determining second boundary information representing the boundary of a second elliptical region based on labeled boundary information, the second elliptical region being a region with the same center and shape as the maximum inscribed ellipse of a labeled rectangular region; determining a first intersection-over-union ratio of the first elliptical region and the second elliptical region based on the first boundary information and the second boundary information; and adjusting the parameters of an initial object recognition model based on the first intersection-over-union ratio to obtain an object recognition model. This can reduce background information interference and improve the recognition accuracy of the object recognition model.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of image processing, and in particular to a training method, an image processing method, and related equipment. BACKGROUND

[0002] Currently, in the training process of an object recognition model based on a neural network, an intersection over union (IoU) between a labeled rectangular region of a sample object and a recognized rectangular region of the sample object recognized by the model is calculated, and the model parameters are adjusted based on the IoU to complete the training.

[0003] However, the four corners of the recognized rectangular region may contain background information. These background information introduces additional noise in the model training process, reducing the recognition accuracy of the trained object recognition model. SUMMARY

[0004] The embodiments of the present application provide a training method, an image processing method, and related equipment, which can reduce the additional noise introduced in the model training process and improve the recognition accuracy of the trained object recognition model.

[0005] In a first aspect, a training method is provided. The method includes: obtaining labeled boundary information of a sample object of a sample image and recognized boundary information of the sample object, the labeled boundary information representing a boundary of a labeled rectangular region in which the sample object is located in the sample image, and the recognized boundary information representing a boundary of a recognized rectangular region of the sample object in the sample image, the shape of the sample object being related to an ellipse; determining first boundary information representing a boundary of a first elliptical region based on the recognized boundary information, the first elliptical region being a region with the same center and the same size as a maximum inscribed ellipse of the recognized rectangular region, and an included angle between a major axis of the first elliptical region and a major axis of the maximum inscribed ellipse of the recognized rectangular region being a target angle; determining second boundary information representing a boundary of a second elliptical region based on the labeled boundary information, the second elliptical region being a region with the same center and the same shape as a maximum inscribed ellipse of the labeled rectangular region, and an included angle between a major axis of the second elliptical region and a major axis of the maximum inscribed ellipse of the labeled rectangular region being the target angle; determining a first intersection over union of the first elliptical region and the second elliptical region based on the first boundary information and the second boundary information; and adjusting parameters of an initial object recognition model based on the first intersection over union to obtain an object recognition model.

[0006] In a possible implementation, the target angle is obtained by: obtaining an image of the identified rectangular region; performing pixel distribution analysis on the image of the identified rectangular region to obtain a main direction of pixel distribution; and calculating an included angle between the main direction and a reference direction to obtain the target angle.

[0007] In a possible implementation, the target angle is 0 degrees.

[0008] In a possible implementation, the first intersection-over-union ratio of the first elliptical region and the second elliptical region is determined based on the first boundary information and the second boundary information, including: determining a minimum circumscribed rectangular region containing the labeled rectangular region and the identified rectangular region based on the labeled boundary information and the identified boundary information; obtaining a set of sampling points containing a target number of sampling points in the minimum circumscribed rectangular region; determining a first number of sampling points in the set that belong to the first elliptical region and belong to the second elliptical region based on the first boundary information and the second boundary information, and determining a second number of sampling points in the set that belong to a union region of the first elliptical region and the second elliptical region based on the first boundary information and the second boundary information; and calculating a ratio of the first number and the second number to obtain the first intersection-over-union ratio.

[0009] In a possible implementation, the labeled boundary information of the sample object of the sample image and the identified boundary information of the sample object are obtained, including: obtaining candidate labeled boundary information of a plurality of candidate objects of the sample image, the candidate labeled boundary information representing boundaries of a candidate labeled rectangular region in which the candidate objects are located in the sample image; processing the sample image by using an initial object identification model to obtain candidate identified boundary information of the plurality of candidate objects, the candidate identified boundary information representing boundaries of a candidate identified rectangular region in which the candidate objects are located in the sample image; determining the labeled boundary information of the sample object from the plurality of candidate labeled boundary information, and determining the identified boundary information of the sample object from the plurality of candidate identified boundary information.

[0010] In a possible implementation, the labeled boundary information of the sample object is determined from the plurality of candidate labeled boundary information, and the identified boundary information of the sample object is determined from the plurality of candidate identified boundary information, including: for each candidate object, determining a second intersection-over-union ratio between the candidate labeled rectangular region and the candidate identified rectangular region based on the candidate identified boundary information and the candidate labeled boundary information.

[0011] The second intersection-over-union ratios corresponding to the plurality of candidate objects are filtered to obtain a target intersection-over-union ratio; candidate recognition boundary information corresponding to the target intersection-over-union ratio in candidate recognition boundary information of the plurality of candidate objects is taken as the recognition boundary information, and candidate annotation boundary information corresponding to the target intersection-over-union ratio in candidate annotation boundary information of the candidate object is taken as the annotation boundary information.

[0012] In a possible implementation, the filtering of the second intersection-over-union ratios corresponding to the plurality of candidate objects to obtain a target intersection-over-union ratio comprises: sorting the plurality of second intersection-over-union ratios in descending order to obtain sorted second intersection-over-union ratios, and taking the first K second intersection-over-union ratios in the sorted second intersection-over-union ratios as the target intersection-over-union ratio, where K is a positive integer greater than 1; or determining a second intersection-over-union ratio greater than or equal to an intersection-over-union ratio threshold from the plurality of second intersection-over-union ratios to obtain the target intersection-over-union ratio.

[0013] In a second aspect, an image processing method is provided. The method comprises: obtaining a to-be-processed image; and identifying the to-be-processed image by using an object recognition model to determine a target object in the to-be-processed image, the shape of the target object being related to an ellipse, the object recognition model being trained by the method of any one of the first aspect.

[0014] In a third aspect, a training apparatus is provided. The apparatus comprises: an obtaining unit configured to obtain annotation boundary information of a sample object of a sample image and recognition boundary information of the sample object, the annotation boundary information representing a boundary of an annotation rectangular region in which the sample object is located in the sample image, and the recognition boundary information representing a boundary of a recognition rectangular region of the sample object in the sample image, the shape of the sample object being related to an ellipse; a processing unit configured to determine, based on the recognition boundary information, first boundary information representing a boundary of a first elliptical region, the first elliptical region being a region with the same center and the same size as a maximum inscribed ellipse of the recognition rectangular region, and an included angle between a major axis of the first elliptical region and a major axis of the maximum inscribed ellipse of the recognition rectangular region being a target angle; determine, based on the annotation boundary information, second boundary information representing a boundary of a second elliptical region, the second elliptical region being a region with the same center and the same shape as a maximum inscribed ellipse of the annotation rectangular region, and an included angle between a major axis of the second elliptical region and a major axis of the maximum inscribed ellipse of the annotation rectangular region being the target angle; and determine a first intersection-over-union ratio of the first elliptical region and the second elliptical region based on the first boundary information and the second boundary information; and an adjusting unit configured to adjust parameters of the initial object recognition model based on the first intersection-over-union ratio to obtain an object recognition model.

[0015] In a fourth aspect, an electronic device is provided, comprising a memory for storing a computer program and a processor for invoking and running the computer program from the memory, so that the electronic device executes the method of any one of the first aspect or the second aspect.

[0016] In a fifth aspect, a computer readable storage medium is provided, which stores a computer program, and the computer program is executed to execute the method of any one of the first aspect or the second aspect.

[0017] In a sixth aspect, a computer program product is provided, which comprises computer program instructions, and the computer program instructions are executed to execute the method of any one of the first aspect or the second aspect.

[0018] In a seventh aspect, a chip is provided, comprising a processor and a data interface, and the processor reads instructions stored on a memory through the data interface to implement the method of any one of the first aspect or the second aspect.

[0019] The beneficial effects of the embodiments of the present application compared with the prior art are that the first boundary information representing the boundary of the first elliptical region is determined by using the obtained recognition boundary information, the second boundary information representing the boundary of the second elliptical region is determined by using the obtained labeling boundary information, the first intersection-over-union ratio of the first elliptical region and the second elliptical region is determined based on the first boundary information and the second boundary information, and the parameters of the initial object recognition model are adjusted based on the first intersection-over-union ratio to obtain the object recognition model. In this way, since the contour of the sample object is more fitted to the ellipse, the model training is performed by using the ellipse intersection-over-union ratio (i.e., the first intersection-over-union ratio) in the model training process, the background information interference is reduced, the model can learn the real features of the sample object in the training process, and the recognition accuracy of the object recognition model obtained by training is improved. BRIEF DESCRIPTION OF DRAWINGS

[0020] Figure 1 is a flowchart of a training method provided by the embodiments of the present application;

[0021] Figure 2 is a schematic diagram of a minimum circumscribed rectangular region in a training method provided by the embodiments of the present application;

[0022] Figure 3 is a schematic diagram of a scene for calculating an ellipse intersection-over-union ratio in a training method provided by the embodiments of the present application;

[0023] Figure 4 is a schematic diagram of a rectangular intersection ratio and an ellipse intersection-over-union ratio in a training method provided by the embodiments of the present application;

[0024] Figure 5 is a flow diagram of a method for image processing provided by an embodiment of the present application;

[0025] Figure 6 is a structural diagram of an image processing apparatus provided by an embodiment of the present application;

[0026] Figure 7 is a structural diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0027] The technical solutions in the embodiments of the present application will be clearly and completely described in combination with the drawings in the embodiments of the present application.

[0028] In the following description, specific details are set forth in connection with the particular structures, techniques, and the like to provide a thorough understanding of the embodiments of the present application. However, persons skilled in the art understand that the embodiments of the present application can be achieved in other embodiments without these specific details. In other cases, well-known systems, devices, circuits, and methods are not described in detail in order not to obscure the description of the present application with unnecessary details.

[0029] It should be understood that when used in the specification and the appended claims of the present application, the term "comprising" indicates the presence of the described features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0030] It should also be understood that the term "and / or" used in the specification and the appended claims of the present application means any combination of one or more of the associated listed items and all possible combinations, and includes these combinations.

[0031] In addition, in the description of the specification and the appended claims of the present application, the terms "first", "second", "third", and the like are only used to distinguish the description, and cannot be understood as indicating or implying relative importance.

[0032] In the present application, the reference "one embodiment" or "some embodiments" and the like means that the specific features, structures or characteristics described in connection with the embodiment are included in one or more embodiments of the present application. Therefore, the statements "in one embodiment", "in some embodiments", "in other some embodiments", "in further some embodiments" and the like appearing in different places in the specification are not necessarily all referring to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized.

[0033] Currently, in a training process of a neural network-based object recognition model, an intersection over union between a labeled rectangular region of a sample object and a recognized rectangular region of the sample object is calculated, and based on the intersection over union, model parameters of the object recognition model are adjusted to complete the training. However, when the shape of the sample object is approximately elliptical, the four corner points of the rectangular region can contain background information. The background information introduces additional noise in the model training process, and reduces the recognition accuracy of the trained object recognition model.

[0034] In an example, taking a screen defect detection scenario as an example, the screen defects are, for example, bright and dark spots, bright and dark groups, etc. A deep learning-based natural image object detection algorithm (for example, you only look once (Yolo), mask region-based convolutional neural network (MaskR-CNN), etc.) is applied to the screen defect detection scenario, an object recognition model for recognizing screen defects can be trained, and the object recognition model is used to detect defects in the screen. In the training process, a rectangular intersection over union is usually used to measure the fitting degree between the recognized rectangular region and the labeled rectangular region of the sample object, and based on this, the model parameters of the initial object recognition model are adjusted to obtain the object recognition model. However, when the rectangular region is used to frame the sample object, the background information is often contained in the four corners of the rectangular region, which introduces additional noise in the training, is not conducive to the convergence of the model, and reduces the recognition accuracy of the trained object recognition model.

[0035] Based on this, the embodiments of the present application provide an image processing method. Since the shapes of the screen defects are more consistent with ellipses, in the model training process, the model parameters of the initial object recognition model are adjusted based on an elliptical intersection over union to complete the training to obtain the object recognition model. In this way, the interference of the background information is reduced, the real features of the sample object can be learned in the training process, and the recognition accuracy of the trained object recognition model is improved.

[0036] The image processing method provided by the present application will be explained and described in detail below.

[0037] Figure 1 is a flowchart of a training method provided by an embodiment of the present application. The training method 100 can be applied to an electronic device, such as a terminal device or a server. The training method 100 is not limited to the specific order of Figure 1 It should be understood that in other embodiments, the order of some steps of the training method 100 can be exchanged with each other according to actual needs, or some steps can be omitted or deleted. For example, Figure 1As shown, the training method 100 includes S101-S105, and each step is explained in detail as follows.

[0038] S101, obtaining annotation boundary information of a sample object of a sample image and recognition boundary information of the sample object, the annotation boundary information representing a boundary of an annotation rectangular region where the sample object is located in the sample image, the recognition boundary information representing a boundary of a recognition rectangular region of the sample object in the sample image, and the shape of the sample object being related to an ellipse.

[0039] The number of the sample images is multiple. One sample image includes at least one sample object.

[0040] In the embodiments of the present application, the annotation boundary information of the sample object of the sample image can be obtained by the electronic device in response to an input operation of the terminal device; or the annotation boundary information of the sample object of the sample image can be obtained by other devices in response to an input operation of a user and then sent to the electronic device. The recognition boundary information of the sample object can be obtained by the terminal device by processing the sample image by using an initial object recognition model; or the recognition boundary information of the sample object can be obtained by other devices by processing the sample image by using the initial object recognition model and then sent to the electronic device.

[0041] It should be understood that the shape of the sample object being related to an ellipse can be understood as that the sample object is an ellipse or an approximately ellipse. The similarity between the shape of the sample object and the shape of the ellipse is greater than or equal to a similarity threshold. The sample object can be an object in the sample image that needs to be focused on in the training process.

[0042] In a possible implementation, the annotation boundary information includes position information of the annotation rectangular region and size information of the annotation rectangular region. The position information of the annotation rectangular region includes coordinates of a center point of the annotation rectangular region; wherein the coordinates of the center point of the annotation rectangular region can be determined by establishing a coordinate system with the top-left vertex of the sample image as the origin (0, 0), the horizontal direction to the right as the x-axis, and the vertical direction upward as the y-axis. The size information of the annotation rectangular region includes the width and the height of the annotation rectangular region.

[0043] Exemplarily, the annotation boundary information can be represented as , wherein represents the coordinates of the center point of the annotation rectangular region; 1 represents the width of the annotation rectangular region, 1 represents the height of the annotation rectangular region.

[0044] In a possible implementation, the identifying boundary information includes identifying position information of the rectangular region and identifying size information of the rectangular region. The identifying position information of the rectangular region includes coordinates of a center point of the rectangular region; wherein the coordinates of the center point of the rectangular region can be determined by establishing a coordinate system with a top-left vertex of the sample image as an origin (0, 0), a horizontal direction to the right as an x-axis, and a vertical direction upward as a y-axis. The identifying size information of the rectangular region includes a width and a height of the rectangular region. The rectangular region is also referred to as a predicted rectangular region.

[0045] Exemplarily, the rectangular region can be represented as , wherein represents the coordinates of the center point of the rectangular region; 2 represents the width of the rectangular region, 2 represents the height of the rectangular region.

[0046] Exemplarily, taking a screen defect detection scenario as an example, the sample image can be a sample screen defect image, and the sample object can be a bright-dark point, a bright-dark spot or the like defect. The labeled rectangular region can be, for example, a rectangular region in which a bright-dark point in the sample screen defect image is located, which is obtained by a human labeling manner. The identified rectangular region can be a rectangular region in which the bright-dark point is located, which is obtained by processing the sample screen defect image by using an initial detection model.

[0047] In a possible implementation, obtaining the labeled boundary information of the sample object of the sample image and the identified boundary information of the sample object includes: obtaining candidate labeled boundary information of a plurality of candidate objects of the sample image, the candidate labeled boundary information representing boundaries of candidate labeled rectangular regions in which the candidate objects are located in the sample image; processing the sample image by using an initial object identification model to obtain candidate identified boundary information of the plurality of candidate objects, the candidate identified boundary information representing boundaries of candidate identified rectangular regions in which the candidate objects are located in the sample image; determining the labeled boundary information of the sample object from the plurality of candidate labeled boundary information, and determining the identified boundary information of the sample object from the plurality of candidate identified boundary information.

[0048] It should be understood that a sample image can include a plurality of candidate objects, and each candidate object corresponds to a candidate labeled boundary information and a candidate identified boundary information. The candidate object can be an object that needs to be focused on and is selected from the sample image by a human.

[0049] The candidate annotation boundary information of the multiple candidate objects in the sample image is obtained by marking the rectangular region where each candidate object is located in the sample image, and the candidate annotation boundary information of each candidate object is obtained. In this marking process, the electronic device can record the coordinates of the four vertices and the center point of the candidate annotation rectangular region to obtain the candidate annotation boundary information. The candidate annotation boundary information includes the position information and size information of the candidate annotation rectangular region. The electronic device can use the initial object recognition model to process the sample image to obtain the candidate recognition boundary information of the candidate object. In this way, the candidate annotation rectangular region and the candidate recognition boundary information of each candidate object can be obtained for the same sample image.

[0050] The recognition boundary information of the sample object can be part or all of the multiple candidate recognition boundary information. The annotation boundary information of the sample object can be the candidate annotation boundary information corresponding to the recognition boundary information of the sample object in the multiple candidate annotation boundary information.

[0051] In a possible implementation, when the number of the multiple candidate recognition boundary information is less than or equal to the first preset number, the multiple candidate recognition boundary information are all used as the recognition boundary information. When the number of the multiple candidate recognition boundary information is greater than the first preset number, a second preset number of candidate recognition boundary information are randomly selected from the multiple candidate recognition boundary information as the recognition boundary information, and the second preset number is less than or equal to the first preset number.

[0052] Based on the above scheme, the problem that the model training process is complex and time-consuming due to too many recognition boundary information and annotation boundary information is avoided, the model training efficiency is improved, and the complexity of model training is reduced.

[0053] In a possible implementation, the multiple candidate recognition boundary information can be screened based on the second intersection-over-union ratio between the candidate annotation rectangular region and the candidate recognition rectangular region of the multiple candidate objects to obtain the recognition boundary information.

[0054] For example, when the number of the multiple candidate recognition boundary information is greater than or equal to the first preset number, the multiple candidate recognition boundary information are screened based on the multiple second intersection-over-union ratios to obtain a second preset number of recognition boundary information. The second preset number is less than the first preset number. The second preset number can be represented by K, and K is a positive integer greater than 1.

[0055] Based on the above scheme, the candidate recognition boundary information is filtered to obtain the recognition boundary information, so as to obtain the labeled boundary information corresponding to the recognition boundary information. In this way, the interference information existing in the multiple candidate recognition boundary information can be eliminated, the reliability and accuracy of the obtained recognition boundary information and labeled boundary information are improved, and then the model can be provided with an accurate learning direction, and the recognition accuracy of the trained object recognition model is improved.

[0056] In a possible implementation, for the candidate recognition boundary information and the candidate labeled boundary information of each candidate object, a second intersection-over-union ratio between the candidate labeled rectangular region and the candidate recognition rectangular region can be determined based on the candidate recognition boundary information and the candidate labeled boundary information.

[0057] It should be understood that the second intersection-over-union ratio is a ratio between an intersection area and a union area of the candidate labeled rectangular region and the candidate recognition rectangular region. The second intersection-over-union ratio represents the degree of overlap between the candidate labeled rectangular region and the candidate recognition rectangular region, and the greater the second intersection-over-union ratio, the more accurate the candidate recognition rectangular region detected by the initial object recognition model.

[0058] Exemplarily, for the candidate recognition boundary information and the candidate labeled boundary information of each candidate object, an intersection area between the candidate labeled rectangular region and the candidate recognition rectangular region can be calculated based on the candidate recognition boundary information and the candidate labeled boundary information, and a sum of an area of the candidate labeled rectangular region and an area of the candidate recognition rectangular region can be calculated based on the candidate recognition boundary information and the candidate labeled boundary information to obtain an area sum value. The area sum value is subtracted by the intersection area to obtain a union area between the candidate labeled rectangular region and the candidate recognition rectangular region. Subsequently, a ratio between the intersection area and the union area is calculated to obtain the second intersection-over-union ratio.

[0059] In a possible implementation, the multiple candidate recognition boundary information is filtered based on the second intersection-over-union ratios between the candidate labeled rectangular regions and the candidate recognition rectangular regions of the multiple candidate objects to obtain the recognition boundary information, including: the second intersection-over-union ratios corresponding to the multiple candidate objects are filtered to obtain a target intersection-over-union ratio; candidate recognition boundary information corresponding to the target intersection-over-union ratio in the candidate recognition boundary information of the multiple candidate objects is taken as the recognition boundary information, and candidate labeled boundary information corresponding to the target intersection-over-union ratio in the candidate labeled boundary information of the candidate object is taken as the labeled boundary information.

[0060] The screening of the second intersection-over-union ratios corresponding to the plurality of candidate objects to obtain the target intersection-over-union ratio can be specifically: removing second intersection-over-union ratios with lower values in the second intersection-over-union ratios to obtain the target intersection-over-union ratio. The second intersection-over-union ratio with a lower value indicates that the overlapping part between the candidate annotation rectangular region and the candidate recognition rectangular region is less, which can be caused by the error factors such as misrecognition and positioning deviation of the initial object recognition model. Removing the second intersection-over-union ratio with a lower value can effectively avoid the interference of the error factors on the subsequent model training, and improve the accuracy of the trained object recognition model.

[0061] Exemplarily, an index of the target intersection-over-union ratio can be obtained, candidate recognition boundary information corresponding to the index in the candidate recognition boundary information of the plurality of candidate objects is taken as the recognition boundary information, and candidate annotation boundary information corresponding to the index in the candidate annotation boundary information of the candidate object is taken as the annotation boundary information.

[0062] In a possible implementation, the screening of the second intersection-over-union ratios corresponding to the plurality of candidate objects to obtain the target intersection-over-union ratio can be implemented in the following manner:

[0063] Method one: the plurality of second intersection-over-union ratios are sorted in descending order to obtain a plurality of sorted second intersection-over-union ratios, and the first K second intersection-over-union ratios in the plurality of sorted second intersection-over-union ratios are taken as the target intersection-over-union ratio.

[0064] Exemplarily, the number of the plurality of second intersection-over-union ratios is 20, and K is 10. The 20 second intersection-over-union ratios can be sorted in descending order, and the first 10 second intersection-over-union ratios in the 20 sorted second intersection-over-union ratios are determined as the target intersection-over-union ratio.

[0065] It should be noted that K is exemplified by 10 in the above example, which is only an example and does not limit the number of the plurality of second intersection-over-union ratios and the specific value of K. In actual application, the value of K can be flexibly set according to actual application, and the specific value of K is not limited in the embodiment of the application.

[0066] Method two: the second intersection-over-union ratios greater than or equal to an intersection-over-union ratio threshold value are determined from the plurality of second intersection-over-union ratios to obtain the target intersection-over-union ratio.

[0067] The intersection-over-union ratio threshold value is set in advance.

[0068] Based on the above scheme, the lower second intersection-over-union ratio in the plurality of second intersection-over-union ratios can be eliminated to obtain the target intersection-over-union ratio, so that the target intersection-over-union ratio can truly and accurately reflect the fitting degree between the real feature region of the sample object and the recognition feature region of the sample object, avoid the interference of accidental factors on the model training, and thus improve the training accuracy.

[0069] In a possible implementation, the first boundary information is determined based on the target angle and the recognition boundary information.

[0070] It should be understood that when the target angle is 0 or the target angle is not considered, the first elliptical region is the maximum inscribed ellipse of the recognition rectangular region. When the target angle is not 0, the first elliptical region is obtained by rotating the maximum inscribed ellipse of the recognition rectangular region around the center of the recognition rectangular region based on the target angle.

[0071] In a possible implementation, the first boundary information is determined based on the target angle and the recognition boundary information.

[0072] In a possible implementation, the target angle can be preset or automatically determined by the electronic device.

[0073] For example, the target angle is 0 degrees.

[0074] Based on the above scheme, when the target angle is 0 degrees, the first elliptical region is the region of the maximum inscribed ellipse of the recognition rectangular region. In this way, the first elliptical region no longer has the background information carried by the vertices of the four corners of the recognition rectangular region, avoiding the interference of the background information on subsequent model training and improving the recognition accuracy of the trained object recognition model.

[0075] In a possible implementation, the target angle is obtained by: obtaining an image of the recognition rectangular region; performing pixel distribution analysis on the image of the recognition rectangular region to obtain a main direction of pixel distribution; and calculating an included angle between the main direction and a reference direction to obtain the target angle.

[0076] For example, principal component analysis can be used to analyze the pixel distribution of the image of the rectangular region to obtain the main direction of the pixel distribution. A two-dimensional rectangular coordinate system is established with the top left corner of the sample image as the origin (0,0), the horizontal direction to the right as the x-axis, and the vertical direction upward as the y-axis. The angle between the main direction of the pixel distribution and the positive x-axis is calculated and used as the target angle.

[0077] Based on the above scheme, the target angle can be determined by the angle between the main direction and the reference direction of the pixel distribution in the identification rectangular region. Then, based on this target angle and the identification boundary information, the first boundary information can be determined. Thus, the first elliptical region indicated by the first boundary information can fit the actual extension direction of the sample object, preserving the pixel information of the core area of ​​the sample object while eliminating the interference of background information from the four vertices of the identification rectangular region, thereby improving the accuracy of determining the region where the sample object is located.

[0078] For example, the image used to identify the rectangular region can be a binary image, which can be referred to as the segmentation mask for identifying the rectangular region. This segmentation mask can be the output of the initial object recognition model when the initial object recognition model is still performing the segmentation task.

[0079] In one possible implementation, it can be In Mapped to the minor axis b of an ellipse, Mapped to the major axis of an ellipse, center point ( Using as the center of the ellipse, the maximum inscribed ellipse of the identification rectangle can be obtained. The equation of this maximum inscribed ellipse of the identification rectangle can be expressed by formula (1):

[0080] Formula (1)

[0081] Where a is the length of the major semi-axis of the largest inscribed ellipse of the identified rectangular region, and b is the length of the minor semi-axis of the largest inscribed ellipse of the identified rectangular region. The first ellipse represented by the first elliptical region can be represented by the following formula (2).

[0082] Formula (2)

[0083] Assuming the target angle is not 0 degrees, the process of determining formula (2) will be explained in detail below.

[0084] Assume the largest inscribed ellipse is rotated counterclockwise around the center of the recognition rectangle by an angle. Let the coordinates of a point on the largest inscribed ellipse before rotation be expressed as ( x , y The coordinates of this point after rotation are represented as ( x', y ′, , , , .

[0085] will be x ′ and y ′ respectively according to the cosine and sine trigonometric function identity expansion to get the relationship between the rotated coordinates x ′, y ′ and the pre-rotated x , y , which can be expressed by the following formula (3) and formula (4):

[0086] Formula (3)

[0087] Formula (4)

[0088] Write formula (3) and (4) in matrix form, from which the following formula (5) about the rotation matrix can be obtained

[0089] Formula (5)

[0090] If there is a matrix , such that , where is the identity matrix and and are not equal to 0, it can be proved that is an invertible matrix. After derivation, the rotation matrix can satisfy formula (6) to formula (8). Wherein:

[0091] Formula (6)

[0092] Formula (7)

[0093] Formula (8)

[0094] Therefore, it can be proved that is a rotation matrix, and the transpose of in formula (5) is obtained as follows formula (9)

[0095] Formula (9)

[0096] Bring formula (9) into the formula (1) of the original ellipse, the rotated ellipse equation can be obtained, which can be expressed as the equation shown in the above formula (2).

[0097] S103, determine, based on the labeled boundary information, second boundary information representing a boundary of a second elliptical region, the second elliptical region being a region identical in shape and center to a maximum inscribed ellipse of the labeled rectangular region, and an included angle between a major axis of the second elliptical region and a major axis of the maximum inscribed ellipse of the labeled rectangular region being a target angle.

[0098] It should be understood that when the target angle is 0 or the target angle is not considered, the second elliptical region is the maximum inscribed ellipse of the labeled rectangular region. When the target angle is not 0, the second elliptical region is obtained by rotating the maximum inscribed ellipse of the labeled rectangular region around the center of the labeled rectangular region based on the target angle.

[0099] In a possible implementation, determining, based on the labeled boundary information, the second boundary information representing the boundary of the second elliptical region comprises: determining, based on the target angle and the labeled boundary information, the second boundary information representing the boundary of the second elliptical region, the second elliptical region being obtained by rotating the maximum inscribed ellipse of the labeled rectangular region around the center of the labeled rectangular region based on the target angle.

[0100] In a possible implementation, the target angle can be preset or automatically determined by the electronic device.

[0101] Exemplarily, the target angle is 0 degrees.

[0102] Based on the above scheme, when the target angle is 0 degrees, the second elliptical region is the region of the maximum inscribed ellipse of the labeled rectangular region. In this way, the second elliptical region no longer has the background information carried by the four corners of the labeled rectangular region, avoiding the interference of these background information on subsequent model training, and improving the recognition accuracy of the trained object recognition model.

[0103] In a possible implementation, the target angle is obtained by: obtaining an image of the recognition rectangular region; performing pixel distribution analysis on the image of the recognition rectangular region to obtain a main direction of pixel distribution; and calculating an included angle between the main direction and a reference direction to obtain the target angle.

[0104] Based on the above scheme, the target angle can be determined based on the included angle between the main direction of the pixel distribution of the recognition rectangular region and the reference direction, so as to determine the second boundary information based on the target angle and the labeled boundary information. In this way, the second elliptical region indicated by the second boundary information can fit the actual extension direction of the recognized sample object, retaining the pixel information of the core region of the sample object and eliminating the interference of the background information brought by the four corners of the labeled rectangular region, thereby improving the accuracy of the determined labeled region of the sample object.

[0105] Exemplarily, the second ellipse represented by the second elliptical region can be represented by formula (10):

[0106] Formula (10)

[0107] Wherein, the derivation process of formula (10) can refer to the derivation process of formula (2) of the first ellipse described above, and the embodiments of the present application will not be repeated here.

[0108] S104, based on the first boundary information and the second boundary information, determining the first intersection and union ratio of the first elliptical region and the second elliptical region.

[0109] In a possible implementation, from the aspect of image rendering, the first elliptical region can be rendered based on the first boundary information, and the second elliptical region can be rendered in the same coordinate plane based on the second boundary information. Then, the pixel area of the intersection region between the first elliptical region and the second elliptical region is calculated, and the pixel area of the first elliptical region and the pixel area of the second elliptical region are calculated. The first intersection and union ratio can be calculated by using formula (11) as follows:

[0110] Formula (11)

[0111] It should be understood that the image is composed of discrete pixel points. The pixel area of the first elliptical region refers to the total number of pixel points located in the first elliptical region. The pixel area of the second elliptical region refers to the total number of pixel points located in the second elliptical region. The pixel area of the intersection region between the first elliptical region and the second elliptical region refers to the total number of pixel points located in both the first elliptical region and the second elliptical region.

[0112] Based on the above scheme, the first boundary information can be converted into a visual graph, and the second boundary information can be converted into a visual graph, so as to directly obtain the pixel area of the intersection region, the pixel area of the first elliptical region and the pixel area of the second elliptical region by using image processing technology, to calculate the first intersection and union ratio, thereby improving the accuracy of the calculated first intersection and union ratio, and further ensuring the reliability of the first intersection and union ratio.

[0113] In one possible implementation, determining the first intersection-union ratio (IUR) of the first and second elliptical regions based on the first and second boundary information can be achieved as follows: Based on the labeled boundary information and the identified boundary information, determine the minimum bounding rectangle region containing the labeled rectangle region and the identified rectangle region; obtain a set of sampling points containing the target number within the minimum bounding rectangle region; based on the first and second boundary information, determine the first number of sampling points in the set that belong to both the first and second elliptical regions, and based on the first and second boundary information, determine the second number of sampling points in the set that belong to the union region of the first and second elliptical regions; calculate the ratio of the first number to the second number to obtain the first IUR.

[0114] For example, Figure 2 The region of the minimum bounding rectangle is shown, which includes both the labeled rectangle region and the identification rectangle region. For example... Figure 2 As shown, a two-dimensional Cartesian coordinate system can be established with the center of the minimum bounding rectangle as the origin, with the horizontal direction to the right as the positive x-axis and the vertical direction upward as the positive y-axis. A set of sampling points containing the number of targets within the region of the minimum bounding rectangle is obtained from the region of the minimum bounding rectangle. The number of targets can be represented by t, and this set can be represented by Point{( (), )...( Let t be used to represent t ≤ 10000.

[0115] The first ellipse represented by the first elliptical region can be represented by the above formula (2). The second ellipse represented by the second elliptical region can be represented by the above formula (10). Point{( (), )...( Substitute the coordinates of each sampling point in the formula (2) and formula (10) respectively. When the coordinates of the sampling points satisfy the condition... ≤1, and If the value is ≤1, it indicates that the sampling point belongs to both the first and second elliptical regions. If the sampling point does not satisfy this condition... ≤1, and does not satisfy When ≤1, it indicates that ( ) does not belong to the second ellipse region, that is, the sampling point belongs to the background region. Taking the target number as 1000 for example, in this way, the first number of sampling points belonging to the first ellipse region and belonging to the second ellipse region in 10000 sampling points is, for example, 4000, and the number of sampling points belonging to the background region is 5000, and the second number of sampling points belonging to the union region of the first ellipse region and the second ellipse region is 10000-5000=5000. The first intersection union ratio is the first number / second number=4000 / 5000=4 / 5.

[0116] Exemplarily, the target number is 10000, and the 10000 points can be all points in the minimum circumscribed rectangular region. Figure 3 The first ellipse region, the second ellipse region, the intersection region and the minimum circumscribed rectangular region are shown. When the first intersection union ratio is 4 / 5, it means that there are a large number of points in 10000 points belonging to the background region. It can also be seen intuitively from FIG. 3 that the proportion of background points is large and the number is large.

[0117] Based on the above scheme, the first intersection union ratio is calculated by the sampling point distribution statistics, which simplifies the process of determining the first intersection union ratio and improves the efficiency of determining the first intersection union ratio.

[0118] It should be noted that for the same sample object, the ellipse intersection union ratio (the first intersection union ratio) and the rectangular intersection union ratio can be different. The following will be explained and described exemplarily in combination with Figure 4 .

[0119] As shown in (a) of FIG. 1, Figure 4 , the rectangular intersection union ratio=(area of intersection region) / (area of identified rectangular region+area of labeled rectangular region-area of intersection region)=0.6954. As shown in (b) of FIG. 1, Figure 4 , the calculated ellipse intersection union ratio is 0.6473. Obviously, the rectangular intersection union ratio and the ellipse intersection union ratio are different.

[0120] The above is an example in which the rectangular intersection union ratio is greater than the ellipse intersection union ratio, which is only an example and does not constitute a limitation on the size relationship between the rectangular intersection union ratio and the ellipse intersection union ratio. In actual application, the ellipse intersection union ratio can also be greater than the rectangular intersection union ratio.

[0121] S105, based on the first intersection union ratio, adjusting the parameters of the initial object recognition model to obtain an object recognition model.

[0122] It should be understood that the first intersection union ratio and the loss value are negatively correlated, that is, the higher the first intersection union ratio, the lower the loss value, and the lower the first intersection union ratio, the higher the loss value.

[0123] In the embodiments of the present application, the first intersection-over-union ratios corresponding to the plurality of sample images can be obtained in the above manner, and the first intersection-over-union ratios are used to adjust the model parameters of the initial object recognition model to obtain the object recognition model.

[0124] For example, the number of the first intersection-over-union ratios is K.

[0125] In the embodiments of the present application, the to-be-processed image can be input into the object recognition model, and the object recognition model can process the to-be-processed image to locate the position of the target object existing in the object recognition model.

[0126] For example, taking the screen defect detection scenario as an example, the to-be-processed image can be a to-be-identified screen image, and the object recognition model can be a screen defect recognition model. After processing the to-be-identified screen image by using the screen defect recognition model, the region where the defect exists can be identified. The target object is a defect, such as a bright-dark spot or a bright-dark group.

[0127] In the training process, the sample screen defect image can be input into the initial object recognition model, and the initial object recognition model can output the recognition boundary information of the defect in one training. Then, based on the recognition boundary information of the defect and the annotation boundary information of the defect, the elliptical intersection-over-union ratio can be determined to quantify the detection result, and the model parameters in the initial object recognition model can be adjusted. After multiple iterations of training, the optimal model is obtained, and the model parameter file of the optimal one-time training is saved. The optimal model file is output as the final model. When the initial object recognition model also performs a segmentation task, the segmentation mask of the defect can be output.

[0128] It should be understood that the training method provided herein can not only be applied to the screen defect detection scenario, but also can be applied to a detection scenario in which the shape of the detected object is related to an ellipse, such as a screw defect detection scenario. The embodiments of the present application do not limit the specific application scenarios.

[0129] The embodiments of the present application provide a training method. The first boundary information representing the boundary of the first elliptical region is determined by using the obtained recognition boundary information, and the second boundary information representing the boundary of the second elliptical region is determined by using the obtained annotation boundary information. The first intersection-over-union ratio of the first elliptical region and the second elliptical region is determined based on the first boundary information and the second boundary information. The parameters of the initial object recognition model are adjusted based on the first intersection-over-union ratio to obtain the object recognition model. In this way, since the outline of the sample object is more consistent with the ellipse, the model training is performed by using the elliptical intersection-over-union ratio (i.e., the first intersection-over-union ratio) in the model training process, the background information interference is reduced, the model can learn the real features of the sample object in the training process, and the recognition accuracy of the object recognition model obtained by training is improved.

[0130] The above describes in detail the training process of the object recognition model. The application process of the object recognition model is described in detail below.

[0131] It should be understood that the electronic device used to train the object recognition model and the electronic device used to apply the object recognition model can be the same electronic device or different electronic devices.

[0132] For example, the electronic device used to train the object recognition model is a server, and the electronic device used to apply the object recognition model is a terminal device. The electronic device used to train the object recognition model can also be a server, and the electronic device used to apply the object recognition model can be a terminal device. Of course, the electronic device used to train the object recognition model and the electronic device used to apply the object recognition model can both be servers, or the electronic device used to train the object recognition model and the electronic device used to apply the object recognition model can both be terminal devices.

[0133] Figure 5 FIG. 1 is a flowchart of an image processing method according to an embodiment of the present application. The image processing method can be applied to an electronic device, and the image processing method includes S101 and S102.

[0134] S101, obtaining a to-be-processed image.

[0135] In the embodiments of the present application, the to-be-processed image can be obtained in response to a user input operation; or the to-be-processed image is downloaded by the electronic device from a network; or the to-be-processed image can be sent to the electronic device by another device.

[0136] S102, identifying the to-be-processed image by using an object recognition model to determine a target object in the to-be-processed image, the shape of the target object being related to an ellipse.

[0137] For example, taking a screen defect detection scenario as an example, the to-be-processed image can be a screen image to be identified, and the object recognition model can be a screen defect recognition model. After processing the screen image to be identified by using the screen defect recognition model, the region where the defect exists in the screen image to be identified is identified.

[0138] The training process of the object recognition model can be referred to Figure 1 The training process of the object recognition model in the corresponding embodiments is not described herein again.

[0139] The embodiment of the present application provides an image processing method. Since the contour of the sample object is more close to an ellipse, the model training is performed by using an ellipse intersection-over-union ratio (i.e., a first intersection-over-union ratio) in the model training process, the interference of background information is reduced, the model can learn the real features of the sample object in the training process, and the identification accuracy of the object recognition model is improved. The object recognition model is used for processing a to-be-processed image, so as to identify a target object in the to-be-processed image, and the accuracy of the determined target object is improved.

[0140] The training method and the image processing method of the embodiment of the present application are described in detail above. Figure 1 And Figure 5 The training method and the image processing method of the embodiment of the present application are described in detail above. Figure 6 And Figure 7 The training method and the image processing method of the embodiment of the present application are described in detail above.

[0141] Figure 6 is a schematic structural diagram of an image processing device provided by the embodiment of the present application. The image processing device 600 can include an acquisition unit 610, a processing unit 620 and an adjustment unit 630.

[0142] In some embodiments, the image processing device 600 can be used to implement the steps of the method shown in Figure 1

[0143] The acquisition unit 610 is configured to acquire annotation boundary information of a sample object of a sample image and recognition boundary information of the sample object, the annotation boundary information represents a boundary of an annotation rectangular region in which the sample object is located in the sample image, the recognition boundary information represents a boundary of a recognition rectangular region of the sample object in the sample image, and the shape of the sample object is related to an ellipse.

[0144] The processing unit 620 is configured to determine first boundary information representing a boundary of a first ellipse region based on the recognition boundary information, the first ellipse region is a region with the same center and the same size as a maximum inscribed ellipse of the recognition rectangular region, and an included angle between a major axis of the first ellipse region and a major axis of the maximum inscribed ellipse of the recognition rectangular region is a target angle; determine second boundary information representing a boundary of a second ellipse region based on the annotation boundary information, the second ellipse region is a region with the same center and the same shape as a maximum inscribed ellipse of the annotation rectangular region, and an included angle between a major axis of the second ellipse region and a major axis of the maximum inscribed ellipse of the annotation rectangular region is the target angle; and determine a first intersection-over-union ratio of the first ellipse region and the second ellipse region based on the first boundary information and the second boundary information.

[0145] ​The adjusting unit 630 is configured to adjust parameters of the initial object recognition model based on the first intersection-over-union ratio to obtain the object recognition model.

[0146] Optionally, the obtaining unit 610 is further configured to obtain the image of the identified rectangular region.

[0147] The processing unit 620 is further configured to perform pixel distribution analysis on the image of the identified rectangular region to obtain a main direction of pixel distribution, and calculate an included angle between the main direction and the reference direction to obtain the target angle.

[0148] Optionally, the target angle is 0 degrees.

[0149] Optionally, the processing unit 620 is further configured to determine a minimum circumscribed rectangular region containing the labeled rectangular region and the identified rectangular region based on the labeled boundary information and the identified boundary information.

[0150] The obtaining unit 610 is further configured to obtain a set of sampling points containing the target number of sampling points in the minimum circumscribed rectangular region.

[0151] The processing unit 620 is further configured to determine a first number of sampling points in the set that belong to the first elliptical region and belong to the second elliptical region based on the first boundary information and the second boundary information, and determine a second number of sampling points in the set that belong to a union region of the first elliptical region and the second elliptical region based on the first boundary information and the second boundary information, and calculate a ratio of the first number and the second number to obtain the first intersection-over-union ratio.

[0152] Optionally, the obtaining unit 610 is further configured to obtain candidate labeled boundary information of a plurality of candidate objects in the sample image, the candidate labeled boundary information representing boundaries of candidate labeled rectangular regions in which the candidate objects are located in the sample image.

[0153] The processing unit 620 is further configured to process the sample image by using the initial object recognition model to obtain candidate identified boundary information of the plurality of candidate objects.

[0154] The labeled boundary information of the sample object is determined from the plurality of candidate labeled boundary information, and the identified boundary information of the sample object is determined from the plurality of candidate identified boundary information.

[0155] Optionally, the processing unit 620 is further configured to, for each candidate object, determine a second intersection-over-union ratio between the candidate annotation bounding information and the candidate recognition bounding information based on the candidate recognition bounding information and the candidate annotation bounding information; filter the second intersection-over-union ratios corresponding to the plurality of candidate objects to obtain a target intersection-over-union ratio; and determine the recognition bounding information as the candidate recognition bounding information corresponding to the target intersection-over-union ratio among the candidate recognition bounding information of the plurality of candidate objects, and determine the annotation bounding information as the candidate annotation bounding information corresponding to the target intersection-over-union ratio among the candidate annotation bounding information of the plurality of candidate objects.

[0156] Optionally, the processing unit 620 is further configured to sort the plurality of second intersection-over-union ratios in descending order to obtain a plurality of sorted second intersection-over-union ratios, and determine the target intersection-over-union ratio as the first K second intersection-over-union ratios among the plurality of sorted second intersection-over-union ratios, where K is a positive integer greater than 1; or determine the target intersection-over-union ratio as the second intersection-over-union ratio greater than or equal to an intersection-over-union ratio threshold among the plurality of second intersection-over-union ratios.

[0157] In some other embodiments, the image processing apparatus 600 can not include the adjusting unit 630. The image processing apparatus 600 can be configured to implement Figure 5 the steps of the method shown in FIG. 6.

[0158] The obtaining unit 610 is configured to obtain a to-be-processed image.

[0159] The processing unit 620 is configured to recognize the to-be-processed image by using an object recognition model to determine a target object in the to-be-processed image, where the shape of the target object is related to an ellipse.

[0160] It should be noted that the image processing apparatus 600 described above is embodied in the form of functional units. The term “unit” herein can be implemented in the form of software and / or hardware, and is not limited in a specific manner.

[0161] For example, the “unit” can be a software program, a hardware circuit, or a combination of both, which implements the above functions. The hardware circuit can include an application specific integrated circuit (ASIC), an electronic circuit, a processor (such as a shared processor, a dedicated processor, or a group processor) and a memory for executing one or more software or firmware programs, and other suitable components for supporting the described functions.

[0162] Therefore, the units of each example described in the embodiments of the present application can be implemented in electronic hardware, or in a combination of computer software and electronic hardware. Whether the functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0163] Figure 7 is a structural schematic diagram of an electronic device 700 provided by an embodiment of the present application. As shown in the figure, the electronic device 700 of this embodiment includes at least one processor 701 (only one processor is shown in the figure), a memory 702, and a computer program 703 stored in the memory 702 and executable on the at least one processor 701, and the processor 701 implements the steps of any method embodiment described above when executing the computer program 703. Figure 7 Figure 7

[0164] Those skilled in the art can understand that Figure 7 The electronic device 700 is only an example and does not constitute a limitation on the electronic device 700, and can include more or fewer components than shown in the figure, or combine certain components, or different components, for example, can also include an input / output device, a network access device, etc.

[0165] The processor 701 can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic components, discrete hardware components, etc. The general-purpose processor can be a microprocessor or can also be any conventional processor.

[0166] ​​The memory 702 can be an internal storage unit of the electronic device 700, such as a hard disk or a memory of the electronic device 700 in some embodiments. The memory 702 can also be an external storage device of the electronic device 700, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) card, a Flash Card, and the like equipped on the electronic device 700 in other embodiments. Further, the memory 702 can include both an internal storage unit and an external storage device of the electronic device 700. The memory 702 is used to store an operating system, an application program, a BootLoader, data, and other programs, such as program codes of a computer program, and the like. The memory 702 can also be used to temporarily store data that has been output or will be output.

[0167] It should be noted that the information interaction, execution process, and the like between the above apparatuses / units are based on the same concept as the method embodiments of the present application, and specific functions and technical effects brought by the same can be referred to the method embodiments part for details, which will not be described herein again.

[0168] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above functional units and modules is taken as an example for illustration, and in actual application, the above functions can be completed by different functional units and modules according to needs, that is, the internal structure of the apparatus is divided into different functional units or modules to complete all or part of the functions described above. Each functional unit and module in the embodiments can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit, and the integrated unit can be realized in the form of hardware or in the form of software functional unit. In addition, the specific names of each functional unit and module are only for easy distinction, and do not limit the protection scope of the present application. The specific working process of the units and modules in the system can refer to the corresponding process in the foregoing method embodiments, which will not be described herein again.

[0169] The embodiments of the present application also provide a computer readable storage medium, which stores a computer program. The computer program is executed by a processor to implement any of the above method embodiments.

[0170] The embodiments of the present application provide a computer program product. When the computer program product is running, any of the above method embodiments can be implemented.

[0171] The integrated unit, if implemented in the form of a software function unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on such understanding, the computer program for instructing the related hardware to complete all or part of the processes in the above-mentioned embodiments can be stored in a computer readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by a processor. The computer program includes computer program code, which can be in the form of source code, object code, executable files or some intermediate forms. The computer readable medium at least includes any entity or device capable of carrying the computer program code to the photographing device / terminal equipment, recording medium, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal and software distribution medium. For example, U disk, mobile hard disk, magnetic disk or optical disk, etc. In some possible implementation manners, the computer readable medium can not be an electrical carrier signal and a telecommunication signal.

[0172] In the above embodiments, the description of each embodiment has its own focus, and the parts not described or recorded in detail in a certain embodiment can be referred to the related description of other embodiments.

[0173] Those skilled in the art can appreciate that the units and algorithm steps of the examples described in combination with the embodiments disclosed herein can be realized by electronic hardware or a combination of computer software and electronic hardware. Whether the functions are realized in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0174] In the embodiments provided in the present application, it should be understood that the disclosed system and method can be implemented in other ways. For example, the above-described system embodiments are only schematic, for example, the division of modules or units is only a logical function division, and there can be another division manner in actual implementation, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed mutual coupling or direct coupling or communication connection can be indirect coupling or communication connection through some interface, device or unit, and can be electrical, mechanical or other forms.

[0175] The units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, that is, may be located in one place, or may also be distributed to multiple network units. Part or all of the units can be selected to achieve the purpose of the embodiment scheme according to actual needs.

[0176] The above embodiments are only used to illustrate the technical solutions of the present application, but not limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that the technical solutions recorded in the foregoing embodiments can still be modified, or some technical features can be replaced by equivalents; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. A training method characterized by, The method comprises the following steps: obtaining annotation boundary information of a sample object in a sample image and recognition boundary information of the sample object, wherein the annotation boundary information represents a boundary of an annotation rectangular region in which the sample object is located in the sample image, and the recognition boundary information represents a boundary of a recognition rectangular region in which the sample object is located in the sample image, and the shape of the sample object is related to an ellipse; wherein, the obtaining of the annotation boundary information of the sample object in the sample image and the recognition boundary information of the sample object comprises: obtaining candidate annotation boundary information of a plurality of candidate objects in the sample image, wherein the candidate annotation boundary information represents a boundary of a candidate annotation rectangular region in which the candidate object is located in the sample image; processing the sample image by using an initial object recognition model to obtain candidate recognition boundary information of the plurality of candidate objects, wherein the candidate recognition boundary information represents a boundary of a candidate recognition rectangular region in which the candidate object is located in the sample image; for the candidate recognition boundary information and the candidate annotation boundary information of each candidate object, determining a second intersection-over-union ratio between the candidate annotation rectangular region and the candidate recognition rectangular region based on the candidate recognition boundary information and the candidate annotation boundary information; sorting a plurality of second intersection-over-union ratios in descending order to obtain a plurality of sorted second intersection-over-union ratios, and taking the first K second intersection-over-union ratios in the plurality of sorted second intersection-over-union ratios as target intersection-over-union ratios, wherein K is a positive integer greater than 1; or determining a second intersection-over-union ratio greater than or equal to an intersection-over-union ratio threshold from the plurality of second intersection-over-union ratios to obtain the target intersection-over-union ratio; taking the candidate recognition boundary information corresponding to the target intersection-over-union ratio in the candidate recognition boundary information of the plurality of candidate objects as the recognition boundary information, and taking the candidate annotation boundary information corresponding to the target intersection-over-union ratio in the candidate annotation boundary information of the candidate object as the annotation boundary information; determining first boundary information representing a boundary of a first ellipse region based on the recognition boundary information, wherein the first ellipse region is a region with the same center and the same size as a maximum inscribed ellipse of the recognition rectangular region, and the included angle between the major axis of the first ellipse region and the major axis of the maximum inscribed ellipse of the recognition rectangular region is a target angle; determining second boundary information representing a boundary of a second ellipse region based on the annotation boundary information, wherein the second ellipse region is a region with the same center and the same shape as a maximum inscribed ellipse of the annotation rectangular region, and the included angle between the major axis of the second ellipse region and the major axis of the maximum inscribed ellipse of the annotation rectangular region is the target angle; determining a first intersection-over-union ratio of the first ellipse region and the second ellipse region based on the first boundary information and the second boundary information; adjusting parameters of the initial object recognition model based on the first intersection-over-union ratio to obtain an object recognition model.

2. The method of claim 1, wherein, The target angle is obtained by the following method: obtaining an image of the recognition rectangular region; performing pixel distribution analysis on the image of the recognition rectangular region to obtain a main direction of pixel distribution; An angle between the main direction and the reference direction is calculated to obtain the target angle.

3. The method of claim 1, wherein, The target angle is 0 degrees.

4. The method according to any one of claims 1 to 3, characterized in that, The first IoU is determined based on the first boundary information and the second boundary information. A minimum circumscribed rectangular region containing the labeled rectangular region and the identified rectangular region is determined based on the labeled boundary information and the identified boundary information. A set of sampling points containing a target number of sampling points in the minimum circumscribed rectangular region is obtained. A first number of sampling points in the set that belong to the first elliptical region and the second elliptical region is determined based on the first boundary information and the second boundary information, and a second number of sampling points in the set that belong to a union region of the first elliptical region and the second elliptical region is determined based on the first boundary information and the second boundary information. A ratio of the first number to the second number is calculated to obtain the first IoU.

5. An image processing method characterized by, The method comprises: An image to be processed is obtained. An object recognition model is used to identify the image to be processed to determine a target object in the image to be processed, the shape of the target object being related to an ellipse, and the object recognition model being trained using the method of any one of claims 1-4.

6. An electronic device, comprising: The electronic device comprises a memory for storing a computer program and a processor for calling and running the computer program from the memory, so that the electronic device executes the method of any one of claims 1-4.

7. A computer readable storage medium characterized by The computer readable storage medium stores a computer program, and when the computer program is executed, the method of any one of claims 1-4 is executed.

Images