Method and device for positioning target object in to-be-detected image and storage medium

Abstract

A method and apparatus for positioning a target object in a to-be-detected image, and a storage medium, the method comprising: determining position transformation information of the target object according to first gravity center information and second gravity center information, the first gravity center information comprising first contour gravity center coordinates of the target object in a template image, and the second gravity center information comprising second contour gravity center coordinates of the target object in the template image; the second gravity center information comprises a second contour gravity center coordinate of the target object in the to-be-detected image; the position transformation information is position transformation information of a target object in the to-be-detected image relative to a target object in the template image; and positioning the target object in the to-be-detected image according to the position transformation information and the position information of the marker in the target object in the template image. Therefore, not only is the intelligent degree of determining the position information of the target object increased, but also the probability of errors caused by manual point taking can be reduced, so that the accuracy of the position transformation information is improved, and the accuracy of positioning the target object in the to-be-detected image can be improved.

Description

Method, device and storage medium for locating target objects in images to be detected Technical Field

[0001] The present application relates to the field of image detection technology, and in particular to a method, device, and storage medium for locating a target object in an image to be detected. Background Art

[0002] With the advancement of industrial technology and image processing, more and more factories are collecting product images to obtain information about the products under inspection. However, due to the instability of the products under inspection, the position and structure of the products under inspection may vary between different images, making it difficult to accurately locate the products within the images.

[0003] Summary of the Invention

[0004] The embodiments of the present application provide a method, device, and storage medium for locating a target object in an image to be detected, which can accurately locate the target object in the image to be detected.

[0005] In a first aspect, a method for locating a target in an image to be detected is provided, the method comprising: determining position transformation information of the target based on first center of gravity information and second center of gravity information, wherein the first center of gravity information includes the first contour center of gravity coordinates of the target in the template image, and the second center of gravity information includes the second contour center of gravity coordinates of the target in the image to be detected; the position transformation information is the position transformation information of the target in the image to be detected relative to the target in the template image; and locating the target in the image to be detected based on the position transformation information and the position information of a marker on the target in the template image.

[0006] In an embodiment of the present application, the position transformation information of the target object is determined by the first center of gravity information and the second center of gravity information. There is no need to manually select points in the template image and the image to be detected to determine the position transformation information of the target object. This not only increases the intelligence level of determining the position transformation information of the target object, but also can reduce the error caused by manual point selection, thereby improving the accuracy of the position transformation information, and further improving the accuracy of locating the target object in the image to be detected.

[0007] In some possible embodiments, the marker is a specific line segment on the contour of the target object, and the target object in the image to be detected is located according to the position transformation information and the position information of the marker on the target object in the template image, including: determining the preset target contour in the image to be detected according to the position transformation information and the position information of the specific line segment of the target object in the template image; determining the target contour in the image to be detected according to the area to be detected in the image to be detected that contains the preset target contour, so as to locate the target object in the image to be detected.

[0008] In the above embodiment, since there is a difference between the position information and structural information of the target object in the image to be detected and the position information and structural information of the target object in the template image, based on the position transformation information and the position information of the specific line segment of the target object in the template image, a preset target contour close to the true contour of the target object in the image to be detected can be obtained in advance; and a target contour with higher accuracy is determined based on the area to be detected containing the preset target contour, which can further improve the accuracy of locating the target object in the image to be detected.

[0009] In some possible implementations, the target contour is determined based on the relationship between the gradient value in the area to be detected and a preset threshold value. In this way, the target contour can be determined conveniently.

[0010] In some possible embodiments, the area to be detected includes multiple sub-areas to be detected; determining the target contour based on the relationship between the gradient value in the area to be detected and the preset threshold includes: determining multiple sub-areas to be detected that are adjacently arranged along the preset target contour based on the preset target contour; determining the target contour based on the relationship between the gradient values ​​in the multiple sub-areas to be detected and the preset threshold.

[0011] In the above embodiment, the gradient value of the area to be detected can be determined more accurately by using a plurality of adjacently arranged sub-areas to be detected, thereby more precisely determining the target contour.

[0012] In some possible implementations, the method for detecting target points in an image further includes: obtaining first contour information of the target object in the template image; and determining the coordinates of the first contour centroid based on the first contour information. Thus, because the first contour information includes the coordinates of the contour of the target object in the template image, the coordinates of the first contour centroid can be accurately determined.

[0013] In some possible embodiments, the target object includes a first target object, and / or a second target object, the first contour information includes first contour sub-information, and / or second contour sub-information, the first contour center-of-gravity coordinates include first contour sub-center-of-gravity coordinates, and / or second contour sub-center-of-gravity coordinates, and determining the first contour center-of-gravity coordinates based on the first contour information includes: determining first width sub-information of the contour of the first target object in the template image and first length sub-information of the contour of the first target object in the template image based on the first contour sub-information; determining the vertex coordinates of the first circumscribed sub-box based on the first width sub-information and the first length sub-information; determining the first contour sub-center-of-gravity coordinates based on the vertex coordinates of the first circumscribed sub-box; and / or determining second width sub-information of the contour of the second target object in the template image and second length sub-information of the contour of the second target object in the template image based on the second contour sub-information; determining the vertex coordinates of the second circumscribed sub-box based on the second width sub-information and the second length sub-information; determining the second contour sub-center-of-gravity coordinates based on the vertex coordinates of the second circumscribed sub-box.

[0014] In the above embodiment, since the first circumscribed sub-frame is determined based on the first width sub-information and the first length sub-information of the contour of the first target object in the template image, the first circumscribed sub-frame can include all the coordinates of the contour of the first target object in the template image. Then, the first contour sub-center coordinates are determined based on the vertex coordinates of the first circumscribed sub-frame, and the first contour sub-center coordinates can be determined more accurately. Similarly, the second contour sub-center coordinates can also be determined more accurately.

[0015] In some possible implementations, the method for detecting an object point in an image to be detected further includes: obtaining second contour information of the object in the image to be detected; and determining the coordinates of the second contour center based on the second contour information. Thus, because the second contour information includes the coordinates of the contour of the object in the image to be detected, the coordinates of the second contour center can be accurately determined.

[0016] In some possible embodiments, the target includes a first target and / or a second target, and the second contour information includes third contour sub-information and / or fourth contour sub-information. Obtaining the second contour information of the target in the image to be detected includes: determining at least one image matching block based on the first target in the template image, the image matching block including at least a portion of the first target; obtaining the third contour sub-information based on the degree of matching between the image matching block and the image to be detected; and / or determining at least one image matching block based on the second target in the template image, the image matching block including at least a portion of the second target; and obtaining the fourth contour sub-information based on the degree of matching between the image matching block and the image to be detected. In this way, because the image matching block contains information about the target in the template image, the second contour information can be determined directly based on the image matching block of the template image without introducing other images, making it more convenient to determine the second contour information.

[0017] In some possible embodiments, the second contour centroid coordinates include the third contour sub-centroid coordinates, and / or the fourth contour sub-centroid coordinates, and determining the second contour centroid coordinates based on the second contour information includes: determining the third width sub-information of the contour of the first target object in the image to be detected and the third length sub-information of the contour of the first target object in the image to be detected based on the third contour sub-information; determining the vertex coordinates of the third circumscribed sub-frame based on the third width sub-information and the second length sub-information; and determining the third contour centroid sub-coordinates based on the vertex coordinates of the third circumscribed sub-frame.

[0018] In the above embodiment, since the third circumscribed sub-frame is determined based on the third width sub-information and the third length sub-information of the contour of the first target object in the image to be detected, the third circumscribed sub-frame can include all the coordinates of the contour of the first target object in the image to be detected. Then, the coordinates of the third contour sub-center of gravity are determined based on the vertex coordinates of the third circumscribed sub-frame, and the coordinates of the third contour sub-center of gravity can be determined more accurately; similarly, the coordinates of the fourth contour sub-center of gravity can also be determined more accurately.

[0019] In some possible implementations, the method for detecting target points in an image further includes: obtaining first contour information of the target object in the template image; and determining position information of a specific line segment of the target object in the template image based on the first contour information and inflection point information of the target object in the template image.

[0020] In the above embodiment, since the area near the inflection point of the target object is prone to deformation or information loss, the position information of the specific line segment determined based on the first contour information and the inflection point information of the target object in the template image can provide more accurate contour information for the image to be detected, which is conducive to locating the target object in the image to be detected.

[0021] In some possible implementations, the position transformation information includes translation information and / or rotation information, and determining the position transformation information of the target object based on the first center of gravity information and the second center of gravity information includes determining the translation information and / or rotation information based on the first contour center of gravity coordinates and the second contour center of gravity coordinates. Thus, by determining the translation information and / or rotation information of the target object based on the first center of gravity information and the second center of gravity information, the translation and / or rotation of the target object can be more accurately determined, thereby more accurately locating the target object.

[0022] In some possible implementations, the translation information includes first translation information and second translation information, and the rotation information includes first rotation information and second rotation information. Determining the translation information and / or rotation information based on the first and second contour barycentric coordinates includes: determining the first translation information and / or first rotation information based on the first and third contour sub-barycentric coordinates; and / or determining the second translation information and / or second rotation information based on the second and fourth contour sub-barycentric coordinates. In this way, the translation information of multiple objects in the detected image can be determined simultaneously, thereby improving image processing efficiency.

[0023] In some possible implementations, the first target object is a cover plate, and the second target object is an adapter plate. The adapter plate is attached to the cover plate, and there is an overlapping area between the adapter plate and the cover plate. In this way, the position transformation information of the cover plate and / or the adapter plate can be determined using the first center of gravity information and the second center of gravity information, eliminating the need to manually select points in the template image and the image to be detected to determine the position transformation information of the cover plate and / or the adapter plate. This not only increases the intelligence level of determining the position transformation information of the cover plate and / or the adapter plate, but also reduces the error caused by manual point selection, thereby improving the accuracy of the position transformation information, and further improving the accuracy of locating the cover plate and / or the adapter plate in the image to be detected.

[0024] In some possible implementations, the outline of the target object in the image to be detected has an obscured area.

[0025] In the above embodiment, when there is an obscured area in the contour of the target object in the image to be detected, more complete contour information of the target object can be obtained in the image to be detected through the position information and position transformation information of the specific line segment of the target object in the template image, which is more conducive to locating the target object in the image to be detected.

[0026] In some possible implementations, the method for detecting a target point in an image to be detected further includes determining a degree of match between the outline of the target object in the image to be detected and the outline of the target object in the template image; and determining second center of gravity information if the degree of match meets a preset condition. This allows discontinuation of determining the second center of gravity coordinates and subsequent processing if the outline of the target object in the image to be detected differs significantly from the outline of the target object in the template image, thereby improving the efficiency of locating the target object in the image to be detected.

[0027] In a second aspect, a device for locating a target in an image to be detected is provided, and the device for locating a target in an image to be detected includes: a determination module, the determination module is used to determine position transformation information of the target based on first center of gravity information and second center of gravity information, wherein the first center of gravity information includes the first contour center of gravity coordinates of the target in the template image, and the second center of gravity information includes the second contour center of gravity coordinates of the target in the image to be detected; the position transformation information is the position transformation information of the target in the image to be detected relative to the target in the template image; and a positioning module, the positioning module is used to locate the target in the image to be detected based on the position transformation information and the position information of the marker on the target in the template image.

[0028] In some possible embodiments, the marker is a specific line segment on the outline of the target object, and the determination module is further used to determine the preset target outline in the image to be detected based on the position transformation information and the position information of the marker on the target object in the template image; the determination module is also used to determine the target outline in the image to be detected based on the area to be detected in the image to be detected that contains the preset target outline, so as to locate the target object in the image to be detected.

[0029] In some possible implementations, the determination module is further configured to determine a target contour based on a relationship between a gradient value in the area to be detected and a preset threshold.

[0030] In some possible embodiments, the area to be detected includes multiple sub-areas to be detected, and the determination module is further used to determine multiple sub-areas to be detected that are adjacently arranged along the preset target contour based on the preset target contour; the determination module is also used to determine the target contour based on the relationship between the gradient values ​​in the multiple sub-areas to be detected and the preset threshold.

[0031] In some possible implementations, the device for locating a target in an image to be detected further includes: an acquisition module for acquiring first contour information of the target in the template image; and a determination module for determining the coordinates of the first contour center of gravity based on the first contour information.

[0032] In some possible embodiments, the target object includes a first target object, and / or a second target object, the first contour information includes first contour sub-information, and / or second contour sub-information, the first contour centroid coordinates include first contour sub-centroid coordinates, and / or second contour sub-centroid coordinates, and the determination module is further used to determine the first width sub-information of the contour of the first target object in the template image and the first length sub-information of the contour of the first target object in the template image based on the first contour sub-information; the determination module is further used to determine the vertex coordinates of the first circumscribed sub-box based on the first width sub-information and the first length sub-information; the determination module is further used to determine the first contour centroid sub-coordinates based on the vertex coordinates of the first circumscribed sub-box; and / or, the determination module is further used to determine the second width sub-information of the contour of the second target object in the template image and the second length sub-information of the contour of the second target object in the template image based on the second contour sub-information; the determination module is further used to determine the vertex coordinates of the second circumscribed sub-box based on the second width sub-information and the second length sub-information; the determination module is further used to determine the second contour sub-centroid coordinates based on the vertex coordinates of the second circumscribed sub-box.

[0033] In some possible implementations, the device for locating a target in an image to be detected further includes: an acquisition module further configured to acquire second contour information of the target in the image to be detected; and a determination module further configured to determine the coordinates of the second contour center of gravity based on the second contour information.

[0034] In some possible implementations, the target object includes a first target object and / or a second target object, the second contour information includes third contour sub-information and / or fourth contour sub-information, the determination module is further configured to determine at least one image matching block based on the first target object in the template image, the image matching block including at least a partial area of ​​the first target object; and the acquisition module is further configured to acquire the third contour sub-information based on a degree of matching between the image matching block and the image to be detected;

[0035] And / or, the determination module is also used to determine at least one image matching block based on the second target object in the template image, and the image matching block includes at least a partial area of ​​the second target object; the acquisition module is also used to obtain fourth contour sub-information based on the matching degree between the image matching block and the image to be detected.

[0036] In some possible embodiments, the second contour centroid coordinates include the third contour sub-centroid coordinates, and / or, the fourth contour sub-centroid coordinates, and the determination module is further used to determine the third width sub-information of the contour of the first target object in the image to be detected and the third length sub-information of the contour of the first target object in the image to be detected based on the second contour information; the determination module is further used to determine the vertex coordinates of the third circumscribed sub-frame based on the third width sub-information and the third length sub-information; the determination module is further used to determine the third contour centroid sub-coordinates based on the vertex coordinates of the third circumscribed sub-frame; and / or, the determination module is further used to determine the fourth width sub-information of the contour of the second target object in the image to be detected and the fourth length sub-information of the contour of the second target object in the image to be detected based on the fourth contour sub-information; the determination module is further used to determine the vertex coordinates of the fourth circumscribed sub-frame based on the fourth width sub-information and the fourth length sub-information; the determination module is further used to determine the fourth contour centroid sub-coordinates based on the vertex coordinates of the fourth circumscribed sub-frame.

[0037] In some possible embodiments, the device for locating the target in the image to be detected also includes: an acquisition module for acquiring first contour information of the target in the template image; and a determination module for determining the position information of a specific line segment of the target in the template image based on the first contour information and the inflection point information of the target in the template image.

[0038] In some possible implementations, the position transformation information includes translation information and / or rotation information, and the determination module is further configured to determine the translation information and / or rotation information based on the first contour barycenter coordinates and the second contour barycenter coordinates.

[0039] In some possible embodiments, the translation information includes first translation information and second translation information, and the rotation information includes first rotation information and second rotation information; the determination module is used to determine the first translation information and / or first rotation information based on the first contour sub-center-of-gravity coordinates and the third contour sub-center-of-gravity coordinates; and / or, the determination module is used to determine the second translation information and / or second rotation information based on the second contour sub-center-of-gravity coordinates and the fourth contour sub-center-of-gravity coordinates.

[0040] In some possible implementations, the first target object is a cover plate, and the second target object is an adapter plate. The adapter plate is attached to the cover plate, and there is an overlapping area between the adapter plate and the cover plate.

[0041] In some possible implementations, the outline of the target object in the image to be detected has an obscured area.

[0042] In some possible implementations, the determination module is further configured to determine a degree of matching between a contour of an object in the image to be detected and a contour of an object in the template image; and the determination module is further configured to determine second center of gravity information when the degree of matching meets preset conditions.

[0043] In a third aspect, a device for locating a target object in an image to be detected is provided, comprising a processor and a memory, the memory being used to store a program, and the processor being used to call and run the program from the memory to execute the method for locating a target object in an image to be detected in the above-mentioned first aspect or any possible implementation of the first aspect.

[0044] In a fourth aspect, a computer-readable storage medium is provided, comprising a computer program, which, when executed on a computer, enables the computer to execute the method for locating a target object in an image to be detected in the above-mentioned first aspect or any possible implementation of the first aspect.

[0045] In a fifth aspect, a computer program product containing instructions is provided, which, when executed by a computer, enables the computer to execute the method for locating a target object in an image to be detected in the above-mentioned first aspect or any possible implementation of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments of the present application. Obviously, the drawings described below are only some implementation methods of the present application. For ordinary technicians in this field, other drawings can be obtained based on the drawings without paying any creative work.

[0047] FIG1 is a schematic diagram of the structure of the system architecture provided by this application;

[0048] FIG2 is a flow chart of a method for locating a target object in an image to be detected disclosed in an embodiment of the present application;

[0049] FIG3 is a schematic diagram of a template image disclosed in an embodiment of the present application;

[0050] FIG4 is a schematic diagram of an image to be detected disclosed in an embodiment of the present application;

[0051] FIG5 is a schematic diagram of an image to be detected including a preset target contour disclosed in an embodiment of the present application;

[0052] FIG6 is a partial enlarged schematic diagram of the A3 area in FIG5 disclosed in the embodiment of the present application;

[0053] FIG7 is a schematic diagram of a first contour centroid disclosed in an embodiment of the present application;

[0054] FIG8 is a schematic structural block diagram of a device for locating a target object in an image to be detected disclosed in an embodiment of the present application;

[0055] FIG9 is a schematic diagram of the hardware structure of an apparatus for locating a target object in an image to be detected according to an embodiment of the present application. DETAILED DESCRIPTION

[0056] The following detailed description of the embodiments of the present application is provided in conjunction with the accompanying drawings and examples. The following detailed description of the embodiments and the accompanying drawings are used to illustrate the principles of the present application, but are not intended to limit the scope of the present application, that is, the present application is not limited to the described embodiments.

[0057] The embodiments of this application are applicable to image processing systems, including but not limited to products based on infrared imaging. The image processing system can be applied to various intelligent manufacturing devices equipped with image processing devices, such as personal computers, computer workstations, smartphones, tablet computers, smart cameras, media consumption devices, wearable devices, set-top boxes, game consoles, augmented reality (AR) and virtual reality (VR) devices, and in-vehicle terminals, etc., although the embodiments disclosed in this application are not limited thereto.

[0058] It should be understood that the specific examples in this article are only intended to help those skilled in the art better understand the embodiments of the present application, and are not intended to limit the scope of the embodiments of the present application.

[0059] It should also be understood that in the various embodiments of the present application, the size of the serial number of each process does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0060] It should also be understood that the various implementation methods described in this specification can be implemented individually or in combination, and the embodiments of the present application are not limited to this.

[0061] Unless otherwise indicated, all technical and scientific terms used in the embodiments of this application have the same meaning as those commonly understood by those skilled in the art in the art of this application. The terms used in this application are only for the purpose of describing specific embodiments and are not intended to limit the scope of this application. The term "and / or" used in this application includes any and all combinations of one or more related listed items.

[0062] With the advancement of industrial technology and image processing, more and more factories are collecting product images to obtain information about the products under inspection. However, due to the instability of the products under inspection, the position and structure of the products under inspection may vary between different images, making it difficult to accurately locate the products within the images.

[0063] In some application scenarios, a comprehensive inspection of the product to be inspected is required during a testing phase, and the target object is the product to be inspected. In other application scenarios, only the main components of the product to be inspected need to be inspected during a testing phase, and the target object is the main components of the product to be inspected.

[0064] For example, the structures of products under inspection vary from batch to batch, leading to discrepancies in the information about the inspected items or key components across different inspection images. Alternatively, the positions of some components within the inspected products may vary, sometimes obscuring key components and causing partial loss of information about these components within the inspection images. Consequently, due to the discrepancies in the positional and structural information of the target objects across different inspection images, it can be difficult to accurately locate the target objects within the inspection images.

[0065] Based on the above considerations, in order to accurately locate the target in the image to be detected, the present application proposes a method for locating the target in the image to be detected, which determines the position transformation information of the target in the image to be detected relative to the target in the template image through the contour center of gravity of multiple targets, and locates the target in the image to be detected according to the position transformation information of the target and the specific line segment of the target.

[0066] In this way, the position transformation information of the target object is determined by the first center of gravity information and the second center of gravity information, and there is no need to manually select points in the template image and the image to be detected to determine the position transformation information of the target object. This not only increases the intelligence level of determining the position transformation information of the target object, but also can reduce the error caused by manual point selection, thereby improving the accuracy of the position transformation information, and then improving the accuracy of locating the target object in the image to be detected.

[0067] In order to better understand the solution of the embodiment of the present application, the following briefly introduces the possible application scenarios of the embodiment of the present application with reference to Figure 1.

[0068] As shown in Figure 1, an embodiment of the present application provides a system architecture 100. In Figure 1, a data acquisition device 160 is used to acquire annotation information and a template image. For the method of detecting an object in an image in an embodiment of the present application, the known annotation information is the first center of gravity information of the object in the template image.

[0069] After collecting the annotation information and the template image, the data collection device 160 stores the template image and the annotation information in the database 130 .

[0070] The above-mentioned target model / rule 101 can be used to implement the method for locating the target object in the image to be detected in the embodiment of the present application. In addition, the target model / rule 101 can be applied to different systems or devices, such as the execution device 110 shown in Figure 1. The execution device 110 can be a terminal, such as a mobile phone terminal, a tablet computer, a laptop computer, etc., or a server or a cloud. In Figure 1, the execution device 110 is configured with an input / output (I / O) interface 112 for interacting with external devices. The user can input data to the I / O interface 112 through the client device 140. The input data can include: the image to be detected input by the client device 140 in the embodiment of the present application.

[0071] In some implementations, the client device 140 and the execution device 110 may be the same device. For example, the client device 140 and the execution device 110 may both be terminal devices.

[0072] In other embodiments, the client device 140 may be a different device from the above-mentioned execution device 110. For example, the client device 140 is a terminal device, and the execution device 110 is a cloud, server, or other device. The client device 140 may interact with the execution device 310 through a communication network with any communication mechanism / communication standard. The communication network may be a wide area network, a local area network, a point-to-point connection, or any combination thereof.

[0073] The computing module 111 of the execution device 110 is configured to process input data (e.g., an image to be detected) received by the I / O interface 112. While the computing module 111 of the execution device 110 is performing calculations and other related processing, the execution device 110 may access data, code, etc. in the data storage system 150 for the corresponding processing, and may also store the data, instructions, etc. obtained from the corresponding processing in the data storage system 150.

[0074] Finally, the I / O interface 112 returns the processing result, such as the positioning result of the target object in the image to be detected, to the client device 140 to provide it to the user.

[0075] In the scenario shown in FIG1 , the user can manually input data, which can be performed through the interface provided by I / O interface 112. Alternatively, client device 140 can automatically send input data to I / O interface 112. If user authorization is required for client device 140 to automatically send input data, the user can set the corresponding permissions in client device 140. The user can view the results output by execution device 110 on client device 140, which can be presented in a display, sound, action, or other specific form. Client device 140 can also serve as a data acquisition terminal, collecting input data input into I / O interface 112 and output results from I / O interface 112 as a new template image and first center of gravity information, and storing them in database 130. Of course, collection can also be performed without client device 140, with I / O interface 112 directly storing the input data input into I / O interface 112 and output results from I / O interface 112 as a new template image and first center of gravity information in database 130.

[0076] It is worth noting that Figure 1 is only a schematic diagram of a system architecture provided by an embodiment of the present application. The positional relationship between the devices, components, modules, etc. shown in the figure does not constitute any limitation. For example, in Figure 1, the data storage system 150 is an external memory relative to the execution device 110. In other cases, the data storage system 150 can also be placed in the execution device 110.

[0077] Below, as shown in FIG2 , the main process of the method for locating a target object in an image to be detected according to an embodiment of the present application is introduced.

[0078] 210. Determine the position transformation information of the target object based on the first center of gravity information and the second center of gravity information, wherein the first center of gravity information includes the first contour center of gravity coordinates of the target object in the template image, and the second center of gravity information includes the second contour center of gravity coordinates of the target object in the image to be detected; the position transformation information is the position transformation information of the target object in the image to be detected relative to the target object in the template image.

[0079] It should be understood that the target object may be a main component with poor consistency in the product to be inspected. Moreover, due to the poor consistency of the main component, the contour of the part will be different in the images of different inspected products. The first center of gravity information may include the coordinates of the first contour center of gravity of the target object in multiple template images; the second center of gravity information may include the coordinates of the second contour center of gravity of the target object in multiple images to be inspected. Moreover, the first contour center of gravity coordinates and the second contour center of gravity coordinates may be two-dimensional coordinates including a horizontal coordinate and a vertical coordinate, or three-dimensional coordinates including a horizontal coordinate, a vertical coordinate and an angle. Among them, the angle may be the angle between multiple first contour centers of gravity, or the angle between multiple second contour centers of gravity.

[0080] For example, FIG3 shows a template image according to an embodiment of the present application. FIG4 shows an image to be detected according to an embodiment of the present application. As shown in FIG3 and FIG4, the first contour barycentric coordinates may be the barycentric coordinates obtained based on the complete contour of the target object 20, or may be the barycentric coordinates obtained based on the contour 28 of the first part of the target object 20. The second contour barycentric coordinates may be the barycentric coordinates of the contour 30 of the second part of the target object 20. It should be noted that the positions of the contour 28 of the first part and the contour 30 of the second part on the target object 20 are substantially the same. However, the contour 28 of the first part is closer to the true contour of the target object 20 than the contour 30 of the second part.

[0081] 220 , locate the target object 20 in the image to be detected 3 according to the position transformation information and the position information of the marker on the target object 20 in the template image 2 .

[0082] The marker on the target object 20 in the template image 2 may be a figure with a relatively fixed shape and position on the target object 20 in the template image 2 , such as a rectangle, circle, etc., or a line with a relatively fixed position on the target object 20 .

[0083] The position information of the target object 20 in the image to be detected 3 can be determined based on the position transformation information and the position information of the marker on the target object 20 in the template image 2, and the target object 20 can be located in the image to be detected 3 based on the position information.

[0084] In an embodiment of the present application, the position transformation information of the target object 20 is determined by the first center of gravity information and the second center of gravity information. There is no need to manually select points in the template image 2 and the image to be detected 3 to determine the position transformation information of the target object 20. This not only increases the intelligence level of determining the position information of the target object 20, but also can reduce the error caused by manual point selection, thereby improving the accuracy of the position transformation information, and further improving the accuracy of locating the target object 20 in the image to be detected 3.

[0085] Optionally, in some embodiments of the present application, as shown in Figures 5 and 6, the marker is a specific line segment 25 on the contour of the target object 20, and the target object 20 in the image to be detected 3 is located according to the position transformation information and the position information of the marker on the target object 20 in the template image 2, including: determining the preset target contour 26 in the image to be detected 3 according to the position transformation information and the position information of the specific line segment 25 of the target object 20 in the template image 2; determining the target contour 27 in the image to be detected 3 according to the area to be detected 29 in the image to be detected that contains the preset target contour 26, so as to locate the target object 20 in the image to be detected 3.

[0086] It should be understood that if the specific line segment 25 of the target 20 in the template image 2 is the complete outline of the target 20 in the template image 2, then the preset target outline 26 is the complete outline of the target 20 in the image 3 to be detected. If the specific line segment 25 of the target 20 in the template image 2 is the outline 28 of the first portion of the target 20 in the template image 2, then the preset target outline 26 is the outline of the portion at the corresponding position on the target 20 in the image 3 to be detected. Furthermore, due to the instability of the image 3 to be detected, there may be some difference between the preset target outline 26 and the actual outline of the target 20. However, based on the pixel distribution of the area 29 to be detected, a contour that is consistent with or close to the actual contour of the target 20 can be determined.

[0087] In the above embodiment, since there is a difference between the position information and structural information of the target object 20 in the image to be detected 3 and the position information and structural information of the target object 20 in the template image 2, based on the position transformation information and the position information of the specific line segment 25 of the target object 20 in the template image 2, a preset target contour 26 near the true contour of the target object 20 in the image to be detected 3 can be obtained in advance; and a target contour 27 with higher accuracy is determined based on the detection area containing the preset target contour 26, which can further improve the accuracy of locating the target object 20 in the image to be detected 3.

[0088] Optionally, in some embodiments of the present application, the target contour 27 is determined based on the relationship between the gradient value and a preset threshold value in the area to be detected 29. In this way, the target contour can be determined conveniently.

[0089] For example, a specific line segment 25 represents the outline of a portion of target object 20 in template image 2. As shown in Figure 6, the gradient value of region 29 to be detected can represent the rate of change between two adjacent pixels within region 29. Furthermore, the larger the gradient value at a corresponding location within the region to be detected, the greater the probability that the location is located at an edge. Therefore, if the rate of change between two adjacent pixels within region 29 is greater than or equal to a preset threshold, it can be determined that the two pixels are located near target outline 27.

[0090] In practical applications, the Laplace algorithm may be used to determine the gradient value of the area to be detected 29 and the target contour 27 .

[0091] It is understandable that the size of the area to be detected 29 will affect the positioning accuracy and positioning speed of the target profile 27 .

[0092] Optionally, in some embodiments of the present application, the area to be detected 29 includes multiple sub-areas to be detected 291, and the target contour 27 is determined based on the relationship between the gradient value and the preset threshold in the area to be detected 29, including: determining multiple sub-areas to be detected 291 arranged adjacent to each other along the preset target contour according to the preset target contour 26; and determining the target contour 27 based on the relationship between the gradient value and the preset threshold in the multiple sub-areas to be detected 291.

[0093] For example, the shape of the sub-region to be detected 291 can be rectangular. Multiple rectangular sub-regions to be detected 291 are adjacently arranged along the preset target contour 26. The size and number of the sub-regions to be detected 291 can be determined based on actual circumstances. The method for determining the target contour 27 based on the gradient values ​​within the multiple sub-regions to be detected 29 can be the same as or different from the above method. For example, a caliper can be used to determine the gradient values ​​within the sub-region to be detected 291 and the target contour 27. By determining the target point in the caliper, the target contour 27 is determined by fitting the multiple target points.

[0094] In the above embodiment, the gradient value of the to-be-detected region 29 can be determined more accurately by using a plurality of adjacently arranged to-be-detected sub-regions 291 , thereby more precisely determining the target contour 27 .

[0095] In embodiments of the present application, various methods can be employed to determine the coordinates of the first contour's centroid. In some implementations of the present application, the method for locating target 20 in image 3 to be detected further includes: obtaining first contour information of target 20 in template image 2; and determining the coordinates of the first contour's centroid based on the first contour information. Thus, because the first contour information includes the coordinates of the contour of target 20 in template image 2, the coordinates of the first contour's centroid can be accurately determined.

[0096] The first contour information may be complete contour information of the target object 20 or contour information of a first portion of the target object 20. Furthermore, the first contour information may be obtained through various contour extraction methods, such as template matching algorithm, neural network, boundary tracking algorithm, etc.

[0097] When the first contour information is obtained, the coordinates of the first contour center can be determined by:

[0098] Example 1: Determine the complete contour of the target object 20 or the minimum circumscribed circle of the first portion of the contour 28 according to the first contour information to determine the first contour centroid coordinates.

[0099] Example 2: Determine the coordinates of the first contour center by determining the complete contour of the target object 20 or the maximum inscribed rectangle of the first portion of the contour 28 based on the first contour information.

[0100] It should be understood that in addition to determining the first contour centroid in the above manner, the first contour centroid may also be determined in other manners.

[0101] FIG7 shows a schematic diagram of a first contour gravity center according to an embodiment of the present application.

[0102] As shown in FIG7 , in some embodiments of the present application, the target object 20 includes a first target object 201 and / or a second target object 202, the first contour information includes first contour sub-information and / or second contour sub-information, the first contour centroid coordinates include first contour sub-centroid coordinates (z1, z2, z3), and / or second contour sub-centroid coordinates, and determining the first contour centroid coordinates according to the first contour information includes: determining the first width sub-information of the contour of the first target object 201 in the template image 2 and the first width sub-information of the contour of the first target object 201 in the template image 2 according to the first contour information. First length sub-information; determining the vertex coordinates of the first circumscribed sub-box based on the first width sub-information and the first length sub-information; determining the first contour sub-center coordinates (z1, z2, z3) based on the vertex coordinates of the first circumscribed sub-box; and / or, determining the second width sub-information of the contour of the second target object 202 in the template image 2 and the second length sub-information of the contour of the second target object 202 in the template image 2 based on the second contour sub-information; determining the vertex coordinates of the second circumscribed sub-box based on the second width sub-information and the second length sub-information; determining the second contour sub-center coordinates based on the vertex coordinates of the second circumscribed sub-box.

[0103] The first circumscribed subframe may be the minimum circumscribed rectangle of the outline of the first target object 201 in the template image 2 , the first width sub-information may be the width W of the minimum circumscribed rectangle, and the first length sub-information may be the length H of the minimum circumscribed rectangle.

[0104] The vertex coordinates of the first circumscribed subframe are determined according to the width W and the length H of the minimum circumscribed rectangle.

[0105] For example, the vertex coordinates (a, b), (c, d), (e, f), and (g, h) of the first circumscribed subframe can be determined by the following formulas: a=-W / 2*Cos(Phi)-H / 2*Sin(Phi) b=-W / 2*Sin(Phi)+H / 2*Cos(Phi) c=W / 2*Cos(Phi)-H / 2*Sin(Phi) d=W / 2*Sin(Phi)+H / 2*Cos(Phi) e=W / 2*Cos(Phi)+H / 2*Sin(Phi) f=W / 2*Sin(Phi)-H / 2*Cos(Phi) g=-W / 2*Cos(Phi)+H / 2*Sin(Phi) h=-W / 2*Sin(Phi)-H / 2*Cos(Phi)

[0106] Wherein, Phi is the inclination angle of the first external sub-frame.

[0107] Two intersecting line segments are determined based on the vertex coordinates (a, b), (c, d), (e, f), and (g, h). The coordinates of the first contour subcenter (z1, z2, z3) are determined based on the intersection of the two intersecting line segments. Here, z1 represents the abscissa of the first contour subcenter, z2 represents the ordinate of the first contour subcenter, and z3 represents the angle of the first contour subcenter. Furthermore, the angle z3 may be equal to Phi.

[0108] In the above embodiment, since the first circumscribed sub-frame is determined based on the width information and length information of the contour of the first target object 201 in the template image 2, the first circumscribed sub-frame can include all the coordinates of the contour of the first target object 201 in the template image 2. The first contour sub-center coordinates (z1, z2, z3) are determined based on the vertex coordinates of the first circumscribed sub-frame, and the first contour sub-center coordinates (z1, z2, z3) can be determined more accurately.

[0109] The method for obtaining the coordinates of the second sub-contour center of gravity may be the same as the method for obtaining the coordinates of the first sub-contour center of gravity, and will not be repeated here.

[0110] It should be noted that the first circumscribed subframe can also be the maximum inscribed rectangle of the outline of the first object 201 in the template image 2. The process of determining the coordinates of the first outline subcenter based on this maximum inscribed rectangle is similar to the above implementation process and will not be repeated here. Furthermore, the second circumscribed subframe can also be the maximum inscribed rectangle of the outline of the second object 202 in the template image 2.

[0111] It should be noted that the method for determining the second contour centroid coordinates may be the same as that for determining the first contour centroid coordinates, or may be different from that for determining the first contour centroid coordinates.

[0112] Optionally, in some embodiments of the present application, the method for locating the target 20 in the image 3 to be detected further includes: obtaining second contour information of the target 20 in the image 3 to be detected; and determining the coordinates of the second contour center of gravity based on the second contour information. Thus, because the second contour information includes the coordinates of the contour of the target in the image 3 to be detected, the coordinates of the second contour center of gravity can be accurately determined.

[0113] Furthermore, the second contour information includes contour information of a second portion of the target object 20 in the image to be detected 3. The second contour information can be obtained by various contour extraction methods, such as a neural network, a boundary tracking algorithm, and the like.

[0114] When the first contour information is obtained, the coordinates of the second contour center can be determined according to the second contour information in the following manner:

[0115] Example 1: Determine the coordinates of the second contour center by determining the minimum circumscribed circle of the contour 30 of the second portion of the target object 20 according to the second contour information.

[0116] Example 2: Determine the coordinates of the second contour centroid of the contour 30 of the second portion of the target object 20 based on the second contour information.

[0117] It should be understood that in addition to extracting the second contour information in the above manner, the second contour information may also be determined in other manners.

[0118] Optionally, in some embodiments of the present application, the target 20 includes a first target 201 and / or a second target 202, and the second contour information includes third contour sub-information and / or fourth contour sub-information. Obtaining the second contour information of the target 20 in the image to be detected 3 includes: determining at least one image matching block based on the first target 201 in the template image 2, the image matching block including at least a portion of the first target 201; obtaining the third contour sub-information based on the degree of matching between the image matching block and the image to be detected; and / or determining at least one image matching block based on the second target 202 in the template image 2, the image matching block including at least a portion of the second target 202; and obtaining the fourth contour sub-information based on the degree of matching between the image matching block and the image to be detected. In this way, since the image matching block contains information about the target 20 in the template image 2, the second contour information can be determined directly based on the image matching block of the template image 2 without introducing other images, making it more convenient to determine the second contour information.

[0119] It should be understood that the image matching block can be an M*M image block, where the value of M can be determined based on actual conditions. Furthermore, the number of image matching blocks can be determined based on the size of the first target 201 or the second target 202 and the size of the image matching block. The first target 201 in all image matching blocks can form the second portion of the contour 30 of the first target 201, or the second target 202 in all image matching blocks can form the second portion of the contour 30 of the second target 202. When multiple image matching blocks are determined, each image matching block can be slid within the image 3 to be detected to traverse the image 3 to be detected. If the degree of match between the image matching block and a portion of the image 3 to be detected is greater than or equal to a threshold, it is determined that the contour of the first target 201 or the second target 202 exists in that region. Thus, second contour information, such as the contour 30 of the second portion, can be determined based on the degree of match between the image matching block and the image 3 to be detected.

[0120] Optionally, in some embodiments of the present application, the second contour centroid coordinates include the third contour sub-centroid coordinates, and / or, the fourth contour sub-centroid coordinates, and determining the second contour centroid coordinates based on the second contour information includes: determining the third width sub-information of the contour of the first target object 201 in the image to be detected 3 and the second length sub-information of the contour of the first target object 201 in the image to be detected 3 based on the second contour information; determining the vertex coordinates of the third circumscribed sub-frame based on the second width sub-information and the second length sub-information; determining the third contour centroid sub-coordinates based on the vertex coordinates of the third circumscribed sub-frame; and / or, determining the fourth width sub-information of the contour of the second target object 202 in the image to be detected 3 and the fourth length sub-information of the contour of the second target object 202 in the image to be detected 3 based on the fourth contour sub-information; determining the vertex coordinates of the fourth circumscribed sub-frame based on the fourth width sub-information and the fourth length sub-information; determining the fourth contour centroid sub-coordinates based on the vertex coordinates of the fourth circumscribed sub-frame.

[0121] The third circumscribed subframe may be the minimum circumscribed rectangle of the outline 30 of the second portion of the first target object 201 in the image 3 to be detected, the second width information may be the width W1 of the minimum circumscribed rectangle, and the second length information may be the length H1 of the minimum circumscribed rectangle.

[0122] The vertex coordinates of the third circumscribed subframe are determined according to the width W1 and the length H1 of the minimum circumscribed rectangle.

[0123] For example, the vertex coordinates (a1, b1), (c1, d1), (e1, f1), and (g1, h1) of the third circumscribed subframe can be determined by the following formulas: a1 = -W1 / 2*Cos(Phi1)–H1 / 2*Sin(Phi1) b1 = -W1 / 2*Sin(Phi1)+H1 / 2*Cos(Phi1) c1 = W1 / 2*Cos(Phi1)–H1 / 2*Sin(Phi1) d1 = W1 / 2*Sin(Phi1)+H1 / 2*Cos(Phi1) e1 = W1 / 2*Cos(Phi1)+H1 / 2*Sin(Phi1) f1 = W1 / 2*Sin(Phi1)–H1 / 2*Cos(Phi1) g1 = -W1 / 2*Cos(Phi1)+H1 / 2*Sin(Phi1) h1=-W1 / 2*Sin(Phi1)–H1 / 2*Cos(Phi1)

[0124] Wherein, Phi1 is the tilt angle of the third external sub-frame.

[0125] Two intersecting line segments are determined according to the vertex coordinates (a1, b1), (c1, d1), (e1, f1), and (g1, h1), and the third contour sub-center is determined according to the intersection point of the two intersecting line segments.

[0126] In the above embodiment, since the third circumscribed sub-frame is determined based on the width information and length information of the contour of the first target object 201 in the image 3 to be detected, the third circumscribed sub-frame can include all the coordinates of the contour of the first target object 20 in the image 3 to be detected. Then, the coordinates of the third contour sub-center of gravity are determined based on the vertex coordinates of the third circumscribed sub-frame, which can more accurately determine the coordinates of the third contour sub-center of gravity.

[0127] The method for obtaining the coordinates of the fourth sub-contour center of gravity may be the same as the method for obtaining the coordinates of the third sub-contour center of gravity, which will not be described in detail here.

[0128] It should be noted that the third circumscribed subframe may also be the maximum inscribed rectangle of the outline of first object 201 in detected image 3. The process of determining the coordinates of the third sub-center of gravity based on this maximum inscribed rectangle is similar to the above-described implementation process and will not be repeated here. Furthermore, the fourth circumscribed subframe may also be the maximum inscribed rectangle of the outline of second object 202 in detected image 3.

[0129] It should be understood that in actual applications, due to the instability of the object to be detected, the position of the target object 20 in the image to be detected 3 is different from the position of the target object 20 in the target image, and the position transformation information can be determined according to actual conditions.

[0130] Optionally, in some embodiments of the present application, the position transformation information includes translation information and / or rotation information, and determining the position transformation information of the target object 20 based on the first center of gravity information and the second center of gravity information includes determining the translation information and / or rotation information based on the first contour center of gravity coordinates and the second contour center of gravity coordinates. Thus, by determining the translation information and / or rotation information of the target object 20 based on the first center of gravity information and the second center of gravity information, the translation and / or rotation of the target object 20 can be more accurately determined, thereby more accurately locating the target object 20.

[0131] Among them, the translation information can be the displacement information of the position of the target object 20 in the image to be detected 3 relative to the position of the target object 20 in the template image 2 in the horizontal direction and / or vertical direction; the rotation information can be the rotation angle of the target object 20 in the image to be detected 3 relative to the target object 20 in the template image 2.

[0132] For example, an affine transformation matrix may be determined based on the first contour barycenter coordinates and the second contour barycenter coordinates to determine translation information and / or rotation information.

[0133] It should be understood that the specific line segment 25 of the target object 20 in the template image 2 may be the outline of any part of the target object 20 in the template image 2 or a line segment of a specific part.

[0134] Optionally, in some embodiments of the present application, the method for locating the target object 20 in the image to be detected 3 also includes: obtaining first contour information of the target object 20 in the template image 2; and determining the position information of a specific line segment 25 of the target object 20 in the template image 2 based on the first contour information and the inflection point information of the target object 20 in the template image 2.

[0135] As shown in Figures 3 to 6, the inflection point information of the target object 20 in the template image 2 may be the coordinates of the inflection points A2, B2, C2, D2, E2, F2, G2, and H2 of the outline of the target object 20. The position information of the specific line segments 25 (A2, B2), (B2, C2), (C2, D2), (E2, F2), (F2, G2), and (G2, H2), for example, the coordinates of the specific line segments 25, is determined based on the outline 28 of the first portion of the target object 20 in the template image 2 and the coordinates of the inflection points.

[0136] In the above embodiment, since the area near the inflection point of the target object 20 is prone to deformation or information loss, the position information of the specific line segment 25 determined based on the first contour information and the inflection point information of the target object 20 in the template image 2 can provide more accurate contour information for the image to be detected 3, which is conducive to locating the target object 20 in the image to be detected 3.

[0137] Optionally, in some embodiments of the present application, the method for locating the target object 20 in the image to be detected 3 further includes: the outline of the target object 20 in the image to be detected 3 has an obscured area.

[0138] It should be understood that, as shown in Figures 3 to 7, when there is an obstructed area in the outline of the target in the image to be detected 3, the outline of the target object 20 in the image to be detected 3 will be incomplete. In this case, the second outline information is usually also incomplete.

[0139] In the above embodiment, when there is an obscured area in the contour of the target object 20 in the image to be detected 3, more complete contour information of the target object 20 can be obtained in the image to be detected 3 through the position information and position transformation information of the specific line segment 25 of the target object 20 in the template image 2, which is more conducive to locating the target object 20 in the image to be detected 3.

[0140] Optionally, in some embodiments of the present application, the method for locating the target 20 in the image to be detected 3 further includes: determining the degree of match between the outline of the target 20 in the image to be detected 3 and the outline of the target 20 in the template image 2; and determining second center of gravity information when the degree of match meets a preset condition. In this way, if the outline of the target 20 in the image to be detected 3 differs significantly from the outline of the target 20 in the template image 2, the determination of the second center of gravity coordinates and subsequent processing steps can be discontinued, thereby improving the efficiency of locating the target 20 in the image to be detected 3.

[0141] The degree of matching can be used to indicate the degree of similarity between the outline of the object 20 in the image to be detected 3 and the outline of the object 20 in the template image 2. For example, the degree of matching can be determined using a template matching algorithm. If the degree of matching is greater than or equal to a preset value, it indicates that the outline of the object 20 in the image to be detected 3 and the outline of the object 20 in the template image 2 are highly similar. In other words, when the outline of the object 20 in the image to be detected 3 is obscured, the missing portion of the outline of the object 20 in the image to be detected 3 is relatively small. In this way, the second center of gravity information can be determined according to the above method.

[0142] If the matching degree is less than the preset value, it can be temporarily determined that the target object 20 in the image to be detected 3 is abnormal, and the next step of judgment is performed.

[0143] It should be noted that the preset conditions and preset values ​​can be determined according to actual conditions. For example, the preset condition can be that the matching degree is less than or equal to the preset value, indicating that the outline of the target object 20 in the image to be detected 3 is highly similar to the outline of the target object 20 in the template image 2.

[0144] Optionally, in some embodiments of the present application, the translation information includes first translation information and / or second translation information, and the rotation information includes first rotation information and / or second rotation information; determining the translation information and / or rotation information based on the first contour center of gravity coordinates and the second contour center of gravity coordinates includes: determining the first translation information and / or first rotation information based on the first contour sub-center of gravity coordinates and the third contour sub-center of gravity coordinates; and / or determining the second translation information and / or second rotation information based on the second contour sub-center of gravity coordinates and the fourth contour sub-center of gravity coordinates.

[0145] For example, the first contour sub-center coordinates and the second contour sub-center coordinates can be used to determine the affine transformation matrix of the first target 201 in the detection image 3 relative to the first target 201 in the template image 2 to obtain the first translation information and the first rotation information; the second contour sub-center coordinates can be used to determine the affine transformation matrix of the second target 202 in the detection image 3 relative to the second target 202 in the template image 2 to obtain the second translation information and the second rotation information.

[0146] Optionally, in some embodiments of the present application, the first target object 201 is a cover plate, the second target object 202 is a transition piece, the transition piece is attached to the cover plate, and there is an overlapping area between the transition piece and the cover plate.

[0147] The cover plate is used to cover the battery cell housing and is usually equipped with electrode terminals for inputting or outputting electrical energy. The adapter plate is used to electrically connect the tabs on the battery assembly to the electrode terminals.

[0148] To facilitate understanding by those skilled in the art, this application also proposes an embodiment of a specific application scenario, for example, a scenario of detecting a switching plate and a cover plate in a battery cell.

[0149] As shown in Figures 2 to 7, the template image 2 can be an image including the adapter sheet 23, the cover plate 21 and the blue film 24. The image to be detected 3 can be an image including the adapter sheet 23, the cover plate 21 and the blue film 24. Moreover, in the image to be detected 3, the blue film 24 covers a partial area of ​​the adapter sheet 23 and the cover plate 21. The target object 20 can be the adapter sheet 23 and the cover plate 21. The first contour information can be the contour information of the portion of the adapter sheet 23 and the cover plate 21. The first contour centroid coordinates include two contour centroid coordinates, which can be the contour centroid coordinates of the adapter sheet 23 determined by the aforementioned method based on the partial contour of the specific position of the adapter sheet 23 in the template image 2, and the contour centroid coordinates of the cover plate 21 determined by the aforementioned method based on the partial contour of the specific position of the cover plate 21. The second contour centroid coordinates also include two contour centroid coordinates, which can be the contour centroid coordinates of the adapter plate 23 determined by the aforementioned method based on the partial contour of the specific position of the adapter plate 23 in the image 3 to be detected, and the contour centroid coordinates of the cover plate 21 determined by the aforementioned method based on the partial contour of the specific position of the cover plate 21. Among them, the contour of the specific position of the adapter plate 23 can be a partial or complete contour of a component on the adapter plate 23 that has a large color difference from the adjacent area; the contour of the specific position of the cover plate 21 can be a partial or complete contour of a component on the cover plate 21 that has a large color difference from the adjacent area (for example, black glue 22). The position information of the specific line segment 25 of the target object 20 in the template image 2 can be the position information of the specific position of the cover plate 21 and the adapter plate 23. Among them, the blue film 24 and the black glue 22 can be used for insulation.

[0150] In practical applications, the first contour information can be stored in advance in the storage medium in .shm file format; the first contour center of gravity coordinates can be stored in advance in the storage medium in .ini file format; the position information of the specific line segment 25 can be stored in advance in the storage medium in .tif file format.

[0151] FIG8 is a schematic structural block diagram of a device for locating a target in an image to be detected disclosed in an embodiment of the present application. As shown in FIG8 , an embodiment of the present application further provides a device for locating a target in an image to be detected, the device 80 comprising: a determination module 81, the determination module 81 being used to determine position transformation information of the target based on first center of gravity information and second center of gravity information, wherein the first center of gravity information includes the first contour center of gravity coordinates of the target in the template image, and the second center of gravity information includes the second contour center of gravity coordinates of the target in the image to be detected; the position transformation information is the position transformation information of the target in the image to be detected relative to the target in the template image; and a positioning module 82, the positioning module 82 being used to locate the target in the image to be detected based on the position transformation information and the position information of the marker on the target in the template image.

[0152] Optionally, in some embodiments of the present application, the marker is a specific line segment on the outline of the target object, and the determination module 81 is also used to determine the preset target outline in the image to be detected based on the position transformation information and the position information of the specific line segment of the target object in the template image; the determination module 81 is also used to determine the target outline in the image to be detected based on the area to be detected in the image to be detected that contains the preset target outline, so as to locate the target object in the image to be detected.

[0153] Optionally, in some embodiments of the present application, the determination module 81 is further configured to determine the target contour according to a relationship between a gradient value in the area to be detected and a preset threshold.

[0154] Optionally, in some embodiments of the present application, the area to be detected includes multiple sub-areas to be detected; the determination module 81 is also used to determine multiple sub-areas to be detected that are adjacently arranged along the preset target contour according to the preset target contour; the determination module 81 is also used to determine the target contour based on the relationship between the gradient values ​​in the multiple sub-areas to be detected and the preset threshold.

[0155] Optionally, in some embodiments of the present application, the device for locating the target in the image to be detected also includes: an acquisition module, which is used to obtain first contour information of the target in the template image; the determination module 81 is also used to determine the coordinates of the first contour center of gravity based on the first contour information.

[0156] Optionally, in some embodiments of the present application, the target object includes a first target object, and / or a second target object, the first contour information includes first contour sub-information, and / or second contour sub-information, the first contour centroid coordinates include first contour sub-centroid coordinates, and / or second contour sub-centroid coordinates, and the determination module 81 is further used to determine the first width sub-information of the contour of the first target object in the template image and the first length sub-information of the contour of the first target object in the template image based on the first contour information; the determination module 81 is further used to determine the vertex coordinates of the first circumscribed sub-box based on the first width sub-information and the first length sub-information; the determination module 81 is further used to determine the first contour sub-centroid coordinates based on the vertex coordinates of the first circumscribed sub-box; and / or, the determination module 81 is further used to determine the second width sub-information of the contour of the second target object in the template image and the second length sub-information of the contour of the second target object in the template image based on the second contour sub-information; the determination module 81 is further used to determine the vertex coordinates of the second circumscribed sub-box based on the second width sub-information and the second length sub-information; the determination module 81 is further used to determine the second contour sub-centroid coordinates based on the vertex coordinates of the second circumscribed sub-box.

[0157] Optionally, in some embodiments of the present application, the acquisition module is further configured to acquire second contour information of the target object in the image to be detected; and the determination module 81 is further configured to determine the coordinates of the second contour center of gravity according to the second contour information.

[0158] Optionally, in some embodiments of the present application, the target object includes a first target object, and / or a second target object, the second contour information includes third contour sub-information, and / or fourth contour sub-information, and the determination module 81 is further used to determine at least one image matching block based on the first target object in the template image, and the image matching block includes at least a partial area of ​​the first target object; the acquisition module is further used to obtain the third contour sub-information based on the matching degree between the image matching block and the image to be detected; and / or, the determination module 81 is further used to determine at least one image matching block based on the second target object in the template image, and the image matching block includes at least a partial area of ​​the second target object; the acquisition module is further used to obtain the fourth contour sub-information based on the matching degree between the image matching block and the image to be detected.

[0159] Optionally, in some embodiments of the present application, the second contour centroid coordinates include the third contour sub-centroid coordinates, and / or, the fourth contour sub-centroid coordinates, and the determination module 81 is also used to determine the third width sub-information of the contour of the first target object in the image to be detected and the third length sub-information of the contour of the first target object in the image to be detected based on the third contour sub-information, and the determination module 81 is also used to determine the vertex coordinates of the third circumscribed sub-frame based on the third width sub-information and the third length sub-information; the determination module 81 is also used to determine the third contour sub-centroid coordinates based on the vertex coordinates of the third circumscribed sub-frame; and / or, the determination module 81 is also used to determine the fourth width sub-information of the contour of the second target object in the image to be detected and the fourth length sub-information of the contour of the second target object in the image to be detected based on the fourth contour sub-information; the determination module is also used to determine the vertex coordinates of the fourth circumscribed sub-frame based on the fourth width sub-information and the fourth length sub-information; the determination module is also used to determine the fourth contour centroid sub-coordinates based on the vertex coordinates of the fourth circumscribed sub-frame.

[0160] Optionally, in some embodiments of the present application, the device for locating the target in the image to be detected also includes: an acquisition module for acquiring first contour information of the target in the template image; and the determination module 81 is also used to determine the position information of a specific line segment of the target in the template image based on the first contour information and the inflection point information of the target in the template image.

[0161] Optionally, in some embodiments of the present application, the position transformation information includes translation information and / or rotation information, and the determination module 81 is further used to determine the translation information and / or rotation information based on the first contour barycenter coordinates and the second contour barycenter coordinates.

[0162] Optionally, in some embodiments of the present application, the translation information includes first translation information and second translation information, and the rotation information includes first rotation information and second rotation information; the determination module 81 is used to determine the first translation information and / or first rotation information based on the first contour sub-center-of-gravity coordinates and the third contour sub-center-of-gravity coordinates; and / or, the determination module 81 is used to determine the second translation information and / or second rotation information based on the second contour sub-center-of-gravity coordinates and the fourth contour sub-center-of-gravity coordinates.

[0163] Optionally, in some embodiments of the present application, the first target object is a cover plate, the second target object is an adapter plate, the adapter plate is attached to the cover plate, and there is an overlapping area between the adapter plate and the cover plate.

[0164] Optionally, in some embodiments of the present application, there is an obscured area in the outline of the target object in the image to be detected.

[0165] Optionally, in some embodiments of the present application, the determination module 81 is also used to determine the degree of matching between the outline of the target object in the image to be detected and the outline of the target object in the template image; the determination module 81 is also used to determine the second center of gravity information when the matching degree meets the preset conditions.

[0166] FIG9 is a schematic diagram of the hardware structure of an apparatus for locating an object in an image to be detected according to an embodiment of the present application. The apparatus 800 for locating an object in an image to be detected shown in FIG9 includes a memory 801, a processor 802, a communication interface 803, and a bus 804. The memory 801, processor 802, and communication interface 803 are interconnected via bus 804.

[0167] Memory 801 can be a read-only memory (ROM), a static storage device, or a random access memory (RAM). Memory 801 can store programs. When the program stored in memory 801 is executed by processor 802, processor 802 and communication interface 803 are used to perform the various steps of the method for locating an object in an image to be detected according to an embodiment of the present application.

[0168] The processor 802 can be a general-purpose central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), a graphics processing unit (GPU) or one or more integrated circuits to execute relevant programs to implement the functions required to be performed by the units in the device for locating the target object in the image to be detected in the embodiment of the present application, or to execute the method for locating the target object in the image to be detected in the embodiment of the present application.

[0169] The processor 802 may also be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the method for locating the target object in the image to be detected in the embodiment of the present application may be completed by hardware integrated logic circuits in the processor 802 or software instructions.

[0170] The processor 802 may also be a general-purpose processor, a digital signal processor (DSP), an ASIC, a field programmable gate array (FPGA), or other programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component. It may implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of the present application. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of the present application may be directly implemented as being executed by a hardware processor, or may be executed by a combination of hardware and software modules in the processor. The software module may be located in a storage medium mature in the art, such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, or the like. The storage medium is located in the memory 801, and the processor 802 reads the information in the memory 801 and, in combination with its hardware, performs the functions required to be performed by the units included in the apparatus for locating an object in an image to be detected according to the embodiments of the present application, or performs the method for locating an object in an image to be detected according to the embodiments of the present application.

[0171] The communication interface 803 uses a transceiver device such as, but not limited to, a transceiver to implement communication between the device 800 and other devices or communication networks. For example, traffic data of an unknown device can be obtained through the communication interface 803.

[0172] The bus 804 may include a path for transmitting information between various components of the device 800 (eg, the memory 801 , the processor 802 , and the communication interface 803 ).

[0173] It should be noted that although the above-mentioned device 800 only shows a memory, a processor, and a communication interface, during the specific implementation process, those skilled in the art will understand that the device 800 may also include other components necessary for normal operation. At the same time, according to specific needs, those skilled in the art will understand that the device 800 may also include hardware components that implement other additional functions. In addition, those skilled in the art will understand that the device 800 may also include only the components necessary to implement the embodiments of the present application, and does not necessarily include all the components shown in Figure 9.

[0174] An embodiment of the present application further provides a computer-readable storage medium storing program code for execution by a device, wherein the program code includes instructions for executing the steps in the above-mentioned method for locating a target object in an image to be detected.

[0175] An embodiment of the present application also provides a computer program product, which includes a computer program stored on a computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer executes the above-mentioned method for locating a target object in an image to be detected.

[0176] The aforementioned computer-readable storage medium may be a transient computer-readable storage medium or a non-transitory computer-readable storage medium.

[0177] It should be noted that the device for locating the target object in the image to be detected, the computer-readable storage medium and the computer program product provided in the above embodiments of the present application can respectively realize the corresponding positioning of the target object in the image to be detected in the aforementioned method embodiments, and have the beneficial effects of the corresponding method embodiments. For related descriptions, please refer to the aforementioned method embodiments and will not be repeated here.

[0178] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the device described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0179] In the description of this application, it should be noted that, unless otherwise specified, "plurality" means more than two. In addition, the terms "first," "second," "third," etc. are used for descriptive purposes only and are not to be understood as indicating or implying relative importance.

[0180] Although the present application has been described with reference to preferred embodiments, various modifications may be made thereto and components may be substituted with equivalents without departing from the scope of the present application. In particular, the various technical features described in the various embodiments may be combined in any manner as long as there are no structural conflicts. The present application is not limited to the specific embodiments disclosed herein, but encompasses all technical solutions within the scope of the claims.

Claims

1. A method for locating a target object in an image to be detected, characterized in that: The method comprises: Determining the position transformation information of the target object according to the first center of gravity information and the second center of gravity information, wherein the first center of gravity information includes the first contour center of gravity coordinates of the target object in the template image, and the second center of gravity information includes the second contour center of gravity coordinates of the target object in the image to be detected; the position transformation information is the position transformation information of the target object in the image to be detected relative to the target object in the template image; The target object in the image to be detected is located according to the position transformation information and the position information of the marker on the target object in the template image.

2. The method according to claim 1, characterized in that: The marker is a specific line segment on the outline of the target object, and locating the target object in the image to be detected according to the position transformation information and the position information of the marker on the target object in the template image includes: Determine a preset target contour in the image to be detected according to the position transformation information and the position information of the specific line segment in the template image; According to the to-be-detected area in the to-be-detected image that contains the preset target contour, the target contour in the to-be-detected image is determined to locate the target object in the to-be-detected image.

3. The method according to claim 2, characterized in that The step of determining the target contour in the image to be detected according to the region to be detected that contains the preset target contour in the image to be detected includes: The target contour is determined according to the relationship between the gradient value in the area to be detected and a preset threshold.

4. The method according to claim 3, characterized in that The area to be detected includes a plurality of sub-areas to be detected, and determining the target contour according to the relationship between the gradient value in the area to be detected and a preset threshold value includes: Determine, according to the preset target contour, a plurality of the sub-areas to be detected that are adjacently arranged along the preset target contour; The target contour is determined according to the relationship between the gradient values ​​in the plurality of sub-regions to be detected and the preset threshold.

5. The method according to any one of claims 1 to 4, characterized in that The method further comprises: Acquire first contour information of the target object in the template image; The coordinates of the center of gravity of the first contour are determined according to the first contour information.

6. The method according to claim 5, characterized in that The target object includes a first target object and / or a second target object, the first contour information includes first contour sub-information and / or second contour sub-information, the first contour centroid coordinates include first contour sub-centroid coordinates and / or second contour sub-centroid coordinates, and determining the first contour centroid coordinates according to the first contour information includes: Determine first width sub-information of the outline of the first target object in the template image and first length sub-information of the outline of the first target object in the template image according to the first outline sub-information; Determine vertex coordinates of a first circumscribed subframe according to the first width sub-information and the first length sub-information; Determine the coordinates of the first contour sub-center according to the vertex coordinates of the first circumscribed sub-frame; and / or, Determine second width sub-information of the outline of the second target object in the template image and second length sub-information of the outline of the second target object in the template image according to the second outline sub-information; Determine vertex coordinates of a second circumscribed subframe according to the second width sub-information and the second length sub-information; The coordinates of the second contour sub-center are determined according to the vertex coordinates of the second circumscribed sub-frame.

7. The method according to any one of claims 1 to 6, characterized in that The method further comprises: Acquire second contour information of the target object in the image to be detected; The coordinates of the center of gravity of the second contour are determined according to the second contour information.

8. The method according to claim 7, characterized in that The target object includes a first target object and / or a second target object, the second contour information includes third contour sub-information and / or fourth contour sub-information, and the acquiring the second contour information of the target object in the image to be detected includes: Determine at least one image matching block according to the first target object in the template image, wherein the image matching block includes at least a partial area of ​​the first target object; acquiring the third contour sub-information according to the matching degree between the image matching block and the image to be detected; and / or, Determine at least one image matching block according to the second target object in the template image, wherein the image matching block includes at least a partial area of ​​the second target object; The fourth contour sub-information is acquired according to the matching degree between the image matching block and the image to be detected.

9. The method according to claim 8, characterized in that The second contour barycenter coordinates include the third contour sub-barycenter coordinates and / or the fourth contour sub-barycenter coordinates, and the determining the second contour barycenter coordinates according to the second contour information includes: Determining third width sub-information of the outline of the first target object in the image to be detected and third length sub-information of the outline of the first target object in the image to be detected according to the third outline sub-information; Determine vertex coordinates of a third circumscribed subframe according to the third width sub-information and the third length sub-information; Determine the third contour centroid sub-coordinates according to the vertex coordinates of the third circumscribed sub-frame; and / or, Determining fourth width sub-information of the contour of the second target object in the image to be detected and fourth length sub-information of the contour of the second target object in the image to be detected according to the fourth contour sub-information; Determine vertex coordinates of a fourth circumscribed subframe according to the fourth width sub-information and the fourth length sub-information; The fourth contour centroid sub-coordinates are determined according to the vertex coordinates of the fourth circumscribed sub-frame.

10. The method according to any one of claims 1 to 9, characterized in that The method further comprises: Acquire first contour information of the target object in the template image; The position information of a specific line segment of the target object in the template image is determined according to the first contour information and the inflection point information of the target object in the template image.

11. The method according to any one of claims 1 to 10, characterized in that The position transformation information includes translation information and / or rotation information, and determining the position transformation information of the target object according to the first center of gravity information and the second center of gravity information includes: The translation information and / or the rotation information is determined according to the first contour barycentric coordinates and the second contour barycentric coordinates.

12. The method according to claim 11, characterized in that The translation information includes first translation information and / or second translation information, and the rotation information includes first rotation information and / or second rotation information; The determining the translation information and / or the rotation information according to the first contour barycentric coordinates and the second contour barycentric coordinates includes: Determine the first translation information and / or the first rotation information according to the first contour sub-center-of-gravity coordinates and the third contour sub-center-of-gravity coordinates; and / or, The second translation information and / or the second rotation information is determined according to the second contour sub-center-of-gravity coordinates and the fourth contour sub-center-of-gravity coordinates.

13. The method according to any one of claims 6 to 12, characterized in that The first target object is a cover plate, and the second target object is an adapter sheet. The adapter sheet is attached to the cover plate, and there is an overlapping area between the adapter sheet and the cover plate.

14. The method according to any one of claims 1 to 13, characterized in that There is an obstructed area in the outline of the target object in the image to be detected.

15. The method according to any one of claims 1 to 14, characterized in that The method further comprises: Determining a matching degree between the contour of the target object in the image to be detected and the contour of the target object in the template image; When the matching degree meets a preset condition, the second center of gravity information is determined.

16. A device for locating a target object in an image to be detected, characterized in that: The device comprises: A determination module, the determination module is used to determine the position transformation information of the target object according to the first center of gravity information and the second center of gravity information, wherein the first center of gravity information includes the first contour center of gravity coordinates of the target object in the template image, and the second center of gravity information includes the second contour center of gravity coordinates of the target object in the image to be detected; the position transformation information is the position transformation information of the target object in the image to be detected relative to the target in the template image; A positioning module is used to locate the target object in the image to be detected according to the position transformation information and the position information of the marker on the target object in the template image.

17. The device according to claim 16, characterized in that The marker is a specific line segment on the outline of the target object, and the determination module is further used to determine the preset target outline in the image to be detected according to the position transformation information and the position information of the marker on the target object in the template image; The determination module is further configured to determine the target contour in the image to be detected according to the region to be detected that contains the preset target contour in the image to be detected, so as to locate the target object in the image to be detected.

18. The device according to claim 17, characterized in that The determination module is also used to determine the target contour according to the relationship between the gradient value in the area to be detected and a preset threshold.

19. The device according to claim 18, characterized in that The area to be detected includes a plurality of sub-areas to be detected, and the determination module is further used to determine, according to the preset target contour, a plurality of sub-areas to be detected that are adjacently arranged along the preset target contour; The determination module is further used to determine the target contour according to the relationship between the gradient values ​​in the plurality of sub-regions to be detected and the preset threshold.

20. The device according to any one of claims 16 to 19, characterized in that The device also includes: The acquisition module is used to acquire the first contour information of the target object in the template image; The determination module is further configured to determine the coordinates of the center of gravity of the first contour according to the first contour information.

21. The device according to claim 20, characterized in that The target object includes a first target object and / or a second target object, the first contour information includes first contour sub-information and / or second contour sub-information, the first contour centroid coordinates include first contour sub-centroid coordinates and / or second contour sub-centroid coordinates, and the determination module is further used to determine first width sub-information of the contour of the first target object in the template image and first length sub-information of the contour of the first target object in the template image according to the first contour sub-information; The determining module is further used to determine the vertex coordinates of the first circumscribed subframe according to the first width sub-information and the first length sub-information; The determining module is further configured to determine the first contour sub-center coordinates according to the vertex coordinates of the first circumscribed sub-frame; and / or, The determining module is further configured to determine second width sub-information of the contour of the second target object in the template image and second length sub-information of the contour of the second target object in the template image according to the second contour sub-information; The determination module is further used to determine the vertex coordinates of the second circumscribed subframe according to the second width sub-information and the second length sub-information; The determination module is further used to determine the coordinates of the second contour sub-center according to the vertex coordinates of the second circumscribed sub-frame.

22. The device according to any one of claims 16 to 21, characterized in that The device also includes: The acquisition module is also used to acquire second contour information of the target object in the image to be detected; The determination module is further used to determine the coordinates of the center of gravity of the second contour according to the second contour information.

23. The device according to claim 22, characterized in that The target object includes a first target object and / or a second target object, the second contour information includes third contour sub-information and / or fourth contour sub-information, and the determination module is further used to determine at least one image matching block according to the first target object in the template image, and the image matching block includes at least a partial area of ​​the first target object; The acquisition module is further used to acquire the third contour sub-information according to the matching degree between the image matching block and the image to be detected; and / or, The determination module is further used to determine at least one image matching block according to the second target object in the template image, wherein the image matching block includes at least a partial area of ​​the second target object; The acquisition module is further used to acquire the fourth contour sub-information according to the matching degree between the image matching block and the image to be detected.

24. The device according to claim 22 or 23, characterized in that The second contour barycenter coordinates include third contour sub-center coordinates and / or fourth contour sub-center coordinates, and the determination module is further used to determine third width sub-information of the contour of the first target object in the image to be detected and third length sub-information of the contour of the first target object in the image to be detected according to the third contour sub-information; The determining module is further used to determine the vertex coordinates of the third circumscribed subframe according to the third width sub-information and the third length sub-information; The determining module is further used to determine the third contour centroid sub-coordinates according to the vertex coordinates of the second circumscribed sub-frame; and / or, The determination module is further configured to determine fourth width sub-information of the contour of the second target object in the image to be detected and fourth length sub-information of the contour of the second target object in the image to be detected according to the fourth contour sub-information; The determining module is further used to determine the vertex coordinates of the fourth circumscribed subframe according to the fourth width sub-information and the fourth length sub-information; The determination module is further configured to determine the fourth contour centroid sub-coordinates according to the vertex coordinates of the fourth circumscribed sub-frame.

25. The device according to any one of claims 16 to 24, characterized in that The device also includes: The acquisition module is used to acquire the first contour information of the target object in the template image; The determination module is further configured to determine position information of a specific line segment of the target object in the template image according to the first contour information and inflection point information of the target object in the template image.

26. The device according to any one of claims 16 to 25, characterized in that The position transformation information includes translation information and / or rotation information, and the determination module is further used to determine the translation information and / or the rotation information according to the first contour barycenter coordinates and the second contour barycenter coordinates.

27. The device according to claim 26, characterized in that The translation information includes first translation information and second translation information, and the rotation information includes first rotation information and second rotation information; the determination module is used to determine the first translation information and / or the first rotation information according to the first contour sub-center of gravity coordinates and the third contour sub-center of gravity coordinates; and / or, The determination module is used to determine the second translation information and / or the second rotation information according to the second contour sub-center-of-gravity coordinates and the fourth contour sub-center-of-gravity coordinates.

28. The device according to any one of claims 21 to 27, characterized in that The first target object is a cover plate, and the second target object is an adapter sheet. The adapter sheet is attached to the cover plate, and there is an overlapping area between the adapter sheet and the cover plate.

29. The device according to any one of claims 16 to 28, characterized in that There is an obstructed area in the outline of the target object in the image to be detected.

30. The device according to any one of claims 16 to 29, characterized in that The determination module is also used to determine the matching degree between the outline of the target object in the image to be detected and the outline of the target object in the template image; The determination module is further configured to determine the second center of gravity information when the matching degree meets a preset condition.

31. A device for locating a target object in an image to be detected, characterized in that: The system comprises a processor and a memory, wherein the memory is used to store a program, and the processor is used to call and run the program from the memory to execute the method according to any one of claims 1 to 15.

32. A computer-readable storage medium, characterized in that: The invention comprises a computer program which, when being run on a computer, causes the computer to execute the method according to any one of claims 1 to 15.

Images