Image processing method and apparatus, and related device

Abstract

Disclosed in embodiments of the present application are an image processing method and apparatus, and a related device, which are applicable to the technical field of computers. The method comprises: obtaining a service image comprising a target object, obtaining a template image, and determining an object detection region from the template image; looking up for a region matching the object detection region in the service image, determining a found region as an initial positioning region, and determining a pixel position transformation relationship on the basis of a pixel position of a pixel in the object detection region and a pixel position of a pixel in the initial positioning region; on the basis of the pixel position transformation relationship, determining a local mapping region corresponding to a local reference region in the service image, and using the local mapping region as a target positioning region; and identifying a foreground region in the target positioning image, and performing edge curve fitting processing on pixels in the foreground region, to obtain an edge curve of the target object. By using the embodiments of the present application, the accuracy and stability of the edge curve obtained by fitting can be improved.

Description

Image processing method, device and related equipment

[0001] This application claims priority to the Chinese patent application filed with the China Patent Office on September 14, 2023, with application number 2023111894721 and application name “Image processing methods, devices and related equipment”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of computer technology, and in particular to transaction processing methods, devices, and related equipment. Background Art

[0003] Currently, when detecting an object in an image (e.g., a product in an image captured in an industrial setting), curve fitting can be performed on the edge curve of the object (e.g., product). Currently, edge curve fitting is primarily achieved by performing edge detection on the object (e.g., product). Common edge detection algorithms utilize the discontinuous nature of grayscale values ​​and segment and determine the edge curve based on grayscale mutations.

[0004] However, the inventors discovered in practice that edge curves determined based on the discontinuous grayscale values ​​of product images are prone to edge misjudgment. For example, if a highlight or dirty area appears on a product, the edge of the highlight or dirty area is easily misjudged as the product edge. For example, if the product color is close to the background color, the grayscale value of the pixel is not prone to sudden changes, making it impossible to correctly determine the product edge. Therefore, when fitting an edge curve for an object (e.g., a product) in an image based on the discontinuous grayscale value, it is easy to obtain an incorrect edge curve, thereby reducing the accuracy and stability of the curve fitting.

[0005] Summary of the Invention

[0006] The embodiments of the present application provide an image processing method, apparatus, and related equipment, which help to improve the accuracy and stability of the edge curve of the target object obtained by fitting.

[0007] On the one hand, an embodiment of the present application provides an image processing method, the method comprising:

[0008] Obtaining a business image containing a target object, obtaining a template image for assisting in locating the target object, and determining an object detection area corresponding to the template object from the template image; the template object refers to an object existing in the template image that has the same object type as the target object, and the area size of the object detection area is smaller than the area size of the business image;

[0009] Searching for an area in the service image that matches the object detection area, determining the found area as the initial positioning area of ​​the target object, and determining a pixel position transformation relationship based on the pixel positions of the pixels in the object detection area and the pixel positions of the pixels in the initial positioning area;

[0010] When determining the local reference area from the template image, a local mapping area corresponding to the local reference area is determined in the business image based on the pixel position transformation relationship, and the local mapping area is used as the target positioning area of ​​the target object; the local mapping area refers to the area obtained by performing pixel position transformation on the pixels in the local reference area based on the pixel position transformation relationship;

[0011] A foreground area corresponding to the target object is identified in the target positioning image corresponding to the target positioning area, and edge curve fitting processing is performed on the pixel points in the foreground area to obtain the edge curve of the target object.

[0012] On the one hand, an embodiment of the present application provides an image processing device, the device comprising:

[0013] An image acquisition module is configured to acquire a business image containing a target object, acquire a template image used to assist in locating the target object, and determine an object detection area corresponding to the template object from the template image; the template object is an object existing in the template image that has the same object type as the target object, and the area size of the object detection area is smaller than the area size of the business image;

[0014] a transformation relationship determination module, configured to search for an area in the service image that matches the object detection area, determine the found area as the initial positioning area of ​​the target object, and determine a pixel position transformation relationship based on the pixel positions of the pixels in the object detection area and the pixel positions of the pixels in the initial positioning area;

[0015] A region mapping module is used to determine, when determining a local reference region from a template image, a local mapping region corresponding to the local reference region in the business image based on a pixel position transformation relationship, and use the local mapping region as a target positioning region for the target object; the local mapping region is the region obtained by performing pixel position transformation on the pixels in the local reference region based on the pixel position transformation relationship;

[0016] The curve fitting module is used to identify the foreground area corresponding to the target object in the target positioning image corresponding to the target positioning area, perform edge curve fitting processing on the pixel points in the foreground area, and obtain the edge curve of the target object.

[0017] On the one hand, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. The computer program is suitable for being loaded and executed by a processor so that a computer device having the processor executes the image processing method provided by the embodiment of the present application.

[0018] In one aspect, embodiments of the present application provide a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the image processing method provided in embodiments of the present application.

[0019] The embodiment of the present application can obtain a template image for assisting in positioning the target object when a business image containing a target object is obtained, and then determine an object detection area from the template image; it should be understood that the area size of the object detection area here is smaller than the area size of the business image; in this way, the embodiment of the present application can quickly determine the found area as the initial positioning area of ​​the target object when finding an area matching the object detection area in the business image, so that the pixel position transformation relationship can be determined based on the pixel position of the pixel points in the object detection area and the pixel position of the pixel points in the initial positioning area; further, the embodiment of the present application can directly determine the local mapping area corresponding to the local reference area in the business image based on the pixel position transformation relationship when determining a local reference area from the template image (the local reference area here refers to the local area of ​​the template object that needs to be focused on and is pre-marked in the template image), and then the local mapping area can be used as the target positioning area of ​​the target object; further, the embodiment of the present application can identify the foreground area corresponding to the target object in the target positioning image corresponding to the target positioning area, and then edge curve fitting processing can be performed on the pixel points in the foreground area to obtain the edge curve of the target object. It can be seen from this that the embodiment of the present application can first quickly determine an initial positioning area in the business image based on the object detection area in a template image, and then determine the pixel position transformation relationship based on the initial positioning area and the object detection area, and perform pixel position transformation on the local area pre-marked in the template image (i.e., the aforementioned local reference area) to obtain the local mapping area corresponding to the local reference area, and then quickly use the local mapping area as the target positioning area located in the business image, so that when the foreground area is subsequently identified in the target positioning image corresponding to the target positioning area, the edge curve can be quickly and accurately determined based on the foreground area. Since the embodiment of the present application performs foreground recognition on the image corresponding to the local area of ​​the target object (i.e., the target positioning image), the influence of the target object's uncertain factors (such as highlight areas and dirty spots) on the foreground recognition process can be reduced from the root, thereby improving the accuracy of foreground area recognition, and then in the process of fitting the edge curve through the pixel points in the foreground area, the accuracy of the fitted edge curve can be improved. In addition, considering that the embodiment of the present application fits the edge curve of a local part of the target object when fitting the edge curve, this means that fewer pixel points will be involved in fitting the edge line of the target object. This not only reduces the situation where there are multiple edge curves that meet the fitting conditions but have large actual differences, but also helps to improve the accuracy and stability of the edge curve of the target object obtained by fitting. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] FIG1 is a schematic structural diagram of an image processing system provided in an embodiment of the present application;

[0021] FIG2 is a schematic diagram of a scenario of an image processing method provided in an embodiment of the present application;

[0022] FIG3 is a flow chart of an image processing method provided in an embodiment of the present application;

[0023] FIG4 is a schematic diagram of a template image provided in an embodiment of the present application;

[0024] FIG5 is a schematic diagram of a window sliding scenario provided by an embodiment of the present application;

[0025] FIG6 is a schematic diagram of an effect of performing area mapping provided by an embodiment of the present application;

[0026] FIG7 is a schematic diagram of an effect of a target foreground recognition network provided by an embodiment of the present application;

[0027] FIG8 is a schematic diagram of the structure of a feature extraction component provided in an embodiment of the present application;

[0028] FIG9 is a schematic diagram of a foreground recognition effect provided by an embodiment of the present application;

[0029] FIG10 is a flow chart of an image processing method provided in an embodiment of the present application;

[0030] FIG11 is a process for determining an edge pixel set according to an embodiment of the present application;

[0031] FIG12 is a schematic diagram of a process for determining an edge curve according to an embodiment of the present application;

[0032] FIG13 is a schematic structural diagram of an image processing device provided in an embodiment of the present application;

[0033] FIG14 is a schematic structural diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0034] The embodiment of the present application provides an image processing scheme, which proposes that an initial positioning area that matches the object detection area in a business image can be quickly determined based on the object detection area in a template image, and then based on the pixel position transformation relationship between the initial positioning area and the object detection area, certain pixel points in the local reference area (for example, the pixel points at the four vertices of the mark display box on the local reference area) can be transformed to obtain a local mapping area corresponding to the local reference area in the business image, and the mapped local mapping area can be collectively referred to as the target positioning area of ​​the target object. When the foreground area where the target object is located is identified in the target positioning image corresponding to the target positioning area, the edge curve of the target object can be quickly fitted by the pixels in the foreground area. It should be understood that since the embodiment of the present application performs foreground recognition on the image corresponding to the target object (i.e., the target positioning image) during foreground recognition through the image processing scheme, the foreground recognition can be performed on the image corresponding to the target object (i.e., the target positioning image). In this way, the influence of the target object's uncertain factors (such as highlight areas and dirty spots) on the foreground recognition process can be reduced, thereby improving the accuracy of foreground area recognition and thus improving the accuracy of the fitted edge curve. In addition, in the process of curve fitting through the image processing scheme, the embodiment of the present application essentially fits the edge curve of the local part of the target object. Therefore, there are smaller pixel points that will participate in fitting to obtain the edge line of the target object. In this way, not only can the situation of having multiple edge curves that meet the fitting conditions but have large actual differences be reduced, but it can also help to improve the accuracy and stability of the edge curve of the target object obtained by fitting, and thus can improve the efficiency of edge curve fitting to a certain extent.

[0035] Please refer to Figure 1, which is a structural diagram of an image processing system provided in an embodiment of the present application. As shown in Figure 1, the image processing system may include a terminal device cluster and a server. As shown in Figure 1, the terminal device cluster may include one or more terminal devices. For example, the multiple terminal devices may specifically include the terminal device 11a, the terminal device 12a, and the terminal device 13a shown in Figure 1. In addition, the server may be the server 200a shown in Figure 1. It is understood that the number of terminal devices and servers included in the image processing system is merely schematic. In actual application scenarios, the corresponding number of terminal devices and servers can be flexibly set according to implementation needs. It should be understood that in an embodiment of the present application, multiple terminal devices in the terminal device cluster (such as the terminal device 11a, the terminal device 12a, and the terminal device 13a) can establish a communication connection with the server through a network (i.e., a medium providing a communication link through a connection method such as a wired or wireless communication link or an optical fiber cable), and then data exchange can be performed between the multiple terminal devices and the server based on the communication connection.

[0036] It is understood that any terminal device in the terminal device cluster (such as terminal device 12a) can run a client, which can be an application that provides local services to users (also known as business objects, operation objects). The server 200a shown in Figure 1 can be a background server corresponding to the client, and the server 200a can run a program for providing resources, service data, and other services to the client. For example, in an embodiment of the present application, the client running on the terminal device can be a client for providing image processing services (i.e., the aforementioned local services), so that users can perform operations such as viewing images after image processing through the client.

[0037] It is understandable that terminal devices (such as terminal device 12a) may include but are not limited to mobile phones, computers, intelligent voice interaction devices, smart home appliances, vehicle-mounted terminals, aircraft, smart speakers, smart home appliances, etc., and are not limited here. The server 200a can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms, and are not limited here.

[0038] It can be understood that the image processing solution involved in the embodiment of the present application can be applied to a computer device. For example, the computer device can be a terminal device in the terminal device cluster shown in Figure 1 above, or it can be the server in Figure 1 above, which is not limited here.

[0039] For example, when the image processing scheme involved in the embodiment of the present application is specifically applied to a terminal device (for example, the terminal device 11a in Figure 1 above), the user can directly execute the above image processing scheme through the terminal device 11a. In this way, when the terminal device 11a finally determines the edge curve of a certain industrial device or a certain product (i.e., the target object) based on the image processing scheme, the target positioning image of the target object and the edge curve of the target object determined in the target positioning image can be quickly displayed on the terminal device 11a, so that the user can view the effect of the edge curve fitted by the image processing scheme.

[0040] For another example, when the image processing scheme involved in the embodiment of the present application is specifically applied to a server (for example, the server 200a in FIG1 above), the user can send a request for determining the edge curve of the target object in the business image to the server 200a through a terminal device (for example, the terminal device 11a), and then the server 200a can execute the above-mentioned image processing scheme after receiving the request to finally determine the edge curve of the target object. Furthermore, the server 200a can send the obtained target positioning image and the edge curve of the target object determined in the target positioning image to the terminal device corresponding to the user (for example, the terminal device 11a), so that the terminal device (for example, the terminal device 11a) can display the received target positioning image and display the determined edge curve of the target object in the target positioning image, so that the user can view the effect of the edge curve fitted by the image processing scheme on the server side in the terminal device.

[0041] Further, referring to FIG. 2 , FIG. 2 is a schematic diagram of a scenario in which an edge curve is obtained by fitting an image processing solution according to an embodiment of the present application. It should be understood that the image processing method involved in obtaining an edge curve by fitting an edge curve using this image processing solution may include at least steps S21 through S24 shown in FIG. 2 .

[0042] Specifically, as shown in FIG2 , a computer device can be used to obtain a template image (e.g., template image 202a shown in FIG2 ) for assisting in positioning the target object when a business image (e.g., business image 201a shown in FIG2 ) containing the target object is obtained. It should be understood that in an embodiment of the present application, a template image refers to an image that can be used to assist in positioning a certain industrial device or a certain type of industrial device in an industrial scenario. Among them, the target object contained in the business image 201a may include at least the target object 211a shown in FIG2 . Similarly, the template object contained in the template image 202a may be the template object 222a shown in FIG2 , and the template object 222a refers to a reference object that exists in the template image 202a and can be used as a reference for positioning when assisting in positioning the target object 211a. It should be understood that in an embodiment of the present application, the area where the positioning frame of the detected template object 222a is located can be determined as the object detection area 221a shown in FIG2 in the template image 202a.

[0043] In other words, since the object detection area 221a can be the maximum circumscribed rectangular area determined by the complete object outline of the entire template object, the object detection area 221a does not contain too much background content. In this way, the interference of the background content in the background area can be reduced to a certain extent during the subsequent auxiliary positioning of the target object. It should be understood that the template object can be an object with the same shape as the target object or an object of the same object type as the target object. For example, the target object here can be any one of a large number of industrial devices mass-produced in an industrial scenario according to the template object in the template image.

[0044] As shown in FIG2 , the size of the object detection area 221a can be d1×d2, where d1 is used to represent the area height of the object detection area 221a, and d2 is used to represent the area width of the object detection area 221a. Further, as shown in FIG2 , the computer device can perform step S21: searching for an area in the business image 201a that matches the object detection area 221a. It should be understood that this specifically refers to searching for an area in the business image 201a that matches the template object 222a in the object detection area 221a. The computer device can then determine the found area (i.e., the found area that matches the template object 222a) as the initial positioning area 212a. For example, as shown in FIG2 , the area size (also referred to as the size) of the initial positioning area 212a is consistent with the size of the object detection area 221a, i.e., it can be d1×d2.

[0045] Furthermore, the computer device may execute step S22: determining a pixel position transformation relationship 203a based on the pixels in the initial positioning area 212a and the pixels in the object detection area 221a. It is understood that the pixel position transformation relationship 203a can be used to indicate the position transformation relationship between the pixels in the initial positioning area 212a and the pixels in the object detection area 221a. The computer device may then execute step S23: determining a local mapping area corresponding to the local reference area 223a in the service image 201a based on the pixel position transformation relationship 203a. In other words, the computer device may use the pixel position transformation relationship 203a to perform pixel position transformation on the local reference area 223a pre-marked in the template image 202a (for example, the four vertices of the marked display box corresponding to the local reference area 223a) to obtain four mapping points corresponding to these four vertices in the local reference area 223a. The area containing the four mapping points may then be collectively referred to as the local mapping area corresponding to the local reference area 223a.

[0046] Among them, the area size of the local reference area 223a marked in the template image 202a is d3×d4, wherein d3 refers to the height of the local reference area 223a, and d4 refers to the width of the local reference area 223a. It can be understood that in the embodiment of the present application, the local mapping area is the mapping area in the business image obtained after the pixel position transformation of the local reference area is performed through the aforementioned pixel position transformation relationship 203a, and the determined local reference area can be determined as the target positioning area 213a shown in Figure 2. The area size (also called size) of the target positioning area 213a is consistent with the size of the local reference area 223a, that is, it can be d3×d4. Then, the computer device can execute step S24: determine the target positioning image 204a based on the target positioning area 213a, identify the foreground area of ​​the target object in the target positioning image 204a, and obtain the edge curve 214a based on the foreground area fitting.

[0047] It is understood that the target positioning image 204a can be an image obtained by cropping the target positioning area 213a in the business image, which means that the size of the target positioning image can be d3×d4. It should be understood that the foreground area identified by the computer device in the target positioning image 204a is the area corresponding to the target object. In addition, the edge curve 214a can be a curve obtained by curve fitting the edge pixels of the target object in the foreground area. For example, the fitted curve can be a straight edge line, such as the edge curve 214a shown in Figure 2.

[0048] It is understandable that the image processing solution for finally obtaining an edge curve (for example, an edge straight line) provided by the present application can specifically include the following three stages. Among them, for the first of these three stages, a template image can be used for coarse positioning in the first stage, and then in the process of coarse positioning, the main area of ​​the target object (such as an industrial device) can be roughly located first, and then a rough adjustment of the position of the main area can be made; in the second stage, a semantic segmentation algorithm can be used to cut out the area at the position corresponding to the area in the template image (that is, the above-mentioned local reference area) from the business image (that is, cut out the target positioning area from the business image) to obtain a target positioning image, and then the target positioning image can be semantically segmented to distinguish the foreground and background in the target positioning image, so that a pure edge curve can be fitted based on the distinguished foreground. In other words, the embodiment of the present application can perform semantic segmentation on the image corresponding to the area in the template image after cutting out the area at the position corresponding to the area in the business image to distinguish the accurate foreground area, and then use the edge pixel points of the target object in the foreground area as the pure edge result (i.e., edge pixel points) obtained by visual measurement in the industrial scene; in the third stage, curve fitting can be performed by a curve fitting processing algorithm (such as a random sampling consistent linear regression (RANSAC) algorithm) to fit the edge curve. Using the image processing scheme involved in the embodiment of the present application, the edge straight line of the target object (e.g., industrial device) in the industrial scene can be accurately located by visual measurement. In this way, not only can the interference caused by factors such as device offset, lighting, dirt, and defects be eliminated, but also the problem of large errors and failure in complex scenes in the existing edge detection method can be solved. In addition, when performing foreground recognition, the embodiment of the present application recognizes the image corresponding to the local part of the target object (i.e., the target positioning image), and its processing speed is much lower than the time of deep semantic segmentation operation using a full-size large image. In this way, cost can be effectively saved when performing foreground recognition. In addition, in the process of generating edge curves through the curve fitting processing method in the embodiment of the present application, the number of parameters required is relatively small, which means that only a small number of edge pixels are actually required in the target positioning image to achieve curve fitting, which can improve the stability and accuracy of the edge curve finally fitted.In addition, the algorithm logic involved in this application is clear, stable and controllable, and the calculation results of each step can be viewed visually, which is convenient for quickly locating problems when the algorithm works abnormally, such as viewing the positioning image in the template image (for example, the image of the template object 222a located in the object detection area 221a can be viewed in the template image 202a), and for example, the initial positioning area determined in the business image can be viewed (for example, the image of the target object 211a roughly located can be viewed in the business image 201a), and for example, the target positioning area in the business image, the foreground area identified in the target positioning image corresponding to the target positioning area, and the edge curve fitted by the pixel points in the foreground area. In addition, the edge curve generated by the embodiment of the present application and the real object have extremely high fitting accuracy, which can achieve an algorithm measurement consistency error that is stable less than 2 pixels and a deviation from the actual value that is stable less than 4 pixel values.

[0049] It is understandable that in the embodiment of the present application, when obtaining a business image or a template image, the required hardware environment may include an imaging machine, a camera (camera device), a light source, and a server for performing depth defect detection and a corresponding software environment, etc., which are not limited here. Among them, the hardware of the server can be, for example, an Intel Xeon 8255C CPU (a central processing unit version) and an NVIDIA Tesla V100 graphics card (a graphics card version). In other words, the image processing solution involved in the embodiment of the present application can be implemented on a server equipped with an Intel Xeon 8255C CPU (a central processing unit version) and an NVIDIA Tesla V100 graphics card (a graphics card version). Among them, the coding language involved in the software environment can be selected according to actual needs, such as Python 3.6.8, and the framework used can also be determined according to actual needs, such as opencv-python (a framework type), numpy (a framework type), scikit-learn (a framework type), etc., which are not limited here.

[0050] The embodiments of the present application can be applied to product measurement scenarios in industrial scenarios (for example, product quality inspection scenarios), that is, the edge curve of the corresponding product to be inspected (i.e., the target object) can be quickly and stably determined through the image processing method described in the embodiments of the present application, and then the product measurement in the product quality inspection scenario can be accurately realized based on the edge curve. If the measurement result (i.e., the detection value) does not meet the requirements, it can be indicated that the product is unqualified. It can be seen that the edge curve of the target object can be accurately fitted by the image processing scheme involved in the embodiments of the present application, so that the accuracy of product measurement of the corresponding product can be improved in the product quality inspection scenario, thereby helping to improve the efficiency of quality management of product quality. Among them, it can be understood that after the embodiment of the present application determines the target positioning area through the pixel position transformation relationship, it can quickly obtain the local area of ​​concern to the user (e.g., quality inspector) in the industrial scenario by regional cutting. In addition, since the size of the local area obtained after cropping becomes smaller, the inference time (i.e., inference time) consumed when applying deep learning for foreground recognition (such as semantic segmentation) will be greatly reduced compared to directly recognizing the entire business image. At the same time, since only the foreground recognition of the local area needs to be performed, this means that in the process of training the deep learning model for recognizing the local area in the business image, the time complexity and video memory occupancy of the deep learning model can also be reduced, thereby greatly improving the availability of product quality inspection in industrial scenarios.

[0051] Optionally, the embodiments of the present application can also be applied to other scenarios where it is necessary to fit the edge curve of the target object. For example, in a special effects processing scenario, the image processing scheme involved in the embodiments of the present application can be used to implement special effects processing for objects such as people and / or animals. For example, the target positioning area of ​​the person and / or animal can be determined in the business image through the embodiments of the present application, and the edge curve of the person and / or animal can be fitted based on the foreground area in the target positioning area. It can be understood that for the same person (i.e., the same target object), different target positioning areas can be determined in the business image based on different local reference areas pre-marked in the template image, and then multiple local areas of the target object can be cut out. In this way, edge curves can be fitted in the multiple local areas by the edge pixels belonging to the foreground area (i.e., belonging to the target object) in each local area, so that multiple edge curves can be fitted. In this way, the embodiments of the present application can also perform special effects processing based on each edge curve in the multiple edge areas obtained by fitting in this special effects processing scenario. For example, a corresponding light effect can be determined based on the edge curve to achieve special effects processing for the business image where the target object is located. In addition, since the fitted edge curve is based on the edge pixel points of the local area of ​​the target object, the number of parameters involved in the curve fitting will be relatively small, so the amount of calculation during curve fitting will be reduced to a certain extent, thereby improving the efficiency of edge curve fitting to a certain extent.

[0052] It can be understood that the embodiments of the present application can be applied to the field of artificial intelligence technology. For example, deep learning can be performed through artificial intelligence technology to train a deep learning network for foreground recognition. For ease of understanding, the embodiments of the present application can collectively refer to the deep learning network for foreground recognition as a foreground recognition network. Then, in an industrial scenario, the foreground recognition network obtained by training can be used to identify the foreground area in the target positioning image (that is, the area corresponding to the foreground in the target positioning image).

[0053] It can be understood that the embodiments of the present application can also be applied to the field of cloud technology. For example, the image processing solution provided by the embodiments of the present application can be executed by a cloud server to ultimately fit the edge curve of the target object in the business image.

[0054] It is understood that the above application scenarios are merely examples and do not limit the application scenarios of the technical solutions provided in the embodiments of this application. The technical solutions of this application can also be applied to other scenarios. For example, those skilled in the art will appreciate that with the evolution of system architecture and the emergence of new business scenarios, the technical solutions provided in the embodiments of this application will also be applicable to similar technical problems.

[0055] Further, please refer to Figure 3, which is a flowchart of an image processing method provided in an embodiment of the present application. The method can be executed by a computer device. The method can include at least the following steps S101-S104.

[0056] Step S101: Acquire a business image containing a target object, acquire a template image for assisting in positioning the target object, and determine an object detection area corresponding to the template object from the template image;

[0057] The template object refers to an object existing in the template image and having the same object type as the target object, and the area size of the object detection area is smaller than the area size of the service image.

[0058] The target object may be an object to be edge detected. The target object may be an object, a person, an animal, etc., without limitation. For example, the target object may be an industrial product, a component (i.e., an industrial device), etc. in an industrial scenario.

[0059] The business image may be an image to be processed that is captured (i.e., collected) and contains a target object. This means that the business image contains a target object to be edge-detected. In other words, the business image may be an image obtained by capturing an image of the target object. It is understood that the target object contained in the business image may be a complete object (such as a complete industrial product) or a partial object within an object (such as a component within a complete industrial product), and this is not limited here.

[0060] The template image can be a positioning reference image for assisting in locating the target object. The positioning reference image can refer to an image that can assist in locating the approximate position of the target object when performing positioning detection on the target object contained in the business image. In other words, the embodiment of the present application can roughly locate the position of the target object in the business image through the template image. The template image includes a template object, and the object type of the template object is consistent with the object type of the target object. In other words, when performing positioning detection on the target object in the business image, it is necessary to obtain a template image corresponding to the template object of the same object type as the target object for positioning. For example, the object type can refer to the part type, product type, item type, etc. of the object, which is not limited here. For example, if the target object is a part corresponding to part type A, the template object can be an object corresponding to the same part type (for example, the module object can be another part corresponding to part type A). Therefore, when the computer device obtains a business image containing the target object, it is necessary to obtain a template image for assisting in locating the target object, that is, it is necessary to use the image containing the template object corresponding to part type A as the template image to assist in locating the target object contained in the business image through the template image. It can be understood that by assisting the positioning of the target object through the template image, the initial positioning area of ​​the target object that matches the object detection area in the template image can be determined more quickly, and then the pixel position transformation relationship between the pixel points in the initial positioning area in the business image and the pixel points in the object detection area in the template image can be constructed. In this way, when there is a mark display box on the template image that is pre-marked by the user (for example, a quality inspector in an industrial scenario) that needs to focus on edge detection, the area corresponding to the mark display box on the template image can be called a local reference area, and then the local reference area and the pixel position transformation relationship can be used to quickly and accurately locate the local area (i.e., the target positioning area) in the target object that needs to be processed (for example, foreground recognition and curve fitting) in the business image.

[0061] Among them, the object detection area can be the area where the template object contained in the template image is located, and the object detection area can be used to match with the business image to assist in locating the area of ​​the target object. It is understandable that the object detection area can contain the complete template object, but will not contain too much background content. Since the template object and the target object are objects of the same object type (for example, part type). Therefore, in one implementable manner, the template object contained in the object detection area can be an object that matches the target object (for example, the template object and the target object can be the same industrial part, or different industrial parts of the same part type), so the area where the template object is located determined from the template image can be called the object detection area. Specifically, the computer device can determine a positioning display frame for locating the template object in the template image, where the positioning display frame can be the object detection frame associated with the template object, so the position of the object detection frame on the template image can be determined as the object detection area. Among them, it is understandable that in an embodiment of the present application, the object detection frame can be used to characterize the area where the main body of the template object located in the template image is located.

[0062] Among them, the object detection frame can be a rectangular frame used to select the template object (specifically, it can be the main body of the template object). It can be understood that the object detection frame can be obtained by manual selection, or it can be automatically marked based on target detection algorithms and other methods, which is not limited here. The timing of determining the object detection frame can be to determine the object detection frame associated with the template object when the template image is acquired, or to determine the object detection frame therein after the template image is acquired, and then determine the location of the object detection frame as the object detection area. In other words, the object detection area can also be understood as an area determined based on the template object in the template image.

[0063] Optionally, a template image may include multiple objects, and an object of the same object type as the target object may be determined from the multiple objects included in the template image as the template object, and an object detection frame may be determined for the template object, so that the area where the object detection frame of the currently determined template object is located (i.e., the location of the object detection frame) may be determined as the object detection area. For example, if a template image includes a rough template area corresponding to parts of multiple part types of a product, and the target object is part type 1, the part of part type 1 in the template image may be determined as the template object, and an object detection frame that completely contains the target object may be further located and determined within the rough template area of ​​the template object, and the location of the object detection frame may be determined as the object detection area.

[0064] It is understood that the size of the object detection area is smaller than the size of the business image, and thus, a matching area can be found in the business image. The area size (also referred to as size) refers to a dimension determined by the number of pixels counted as length and width. For example, the area size of a rectangular area refers to the number of pixels corresponding to the length of the rectangular area and the number of pixels corresponding to the width of the rectangular area.

[0065] For example, please refer to Figure 4, which is a schematic diagram of a template image provided in an embodiment of the present application. As shown in Figure 4, the template image 41a may include a template object 411a, and an object detection area 412a determined based on the template object 411a. It can be seen that the object detection area 412a can completely contain the template object 411a and does not contain too much background area, that is, the object detection area 412a is determined by the object detection frame located and detected in the template image 41a for completely characterizing the main position of the template object 411a. The background area here refers to the area where other objects (for example, non-main objects) other than the main body of the object (such as the template object) in the image (such as the template image) are located.

[0066] Specifically, in some possible implementations, if the template image contains multiple objects, and the multiple objects here can specifically include the template object selected in the template image for assisting in positioning the target object, then at this time, in the object detection area 412a shown in Figure 4, the area where the main body of the template object 411a is located (for example, the grayscale area shown in Figure 4) can be called the foreground area, so the other areas in the object detection area 412a except the area where the main body of the template object 411a is located (for example, the grayscale area shown in Figure 4) (for example, the blank area or white space area shown in Figure 4) can be called the background area.

[0067] It is understandable that, generally speaking, the image acquisition of the template image and the image acquisition of the business image are acquired in the same image acquisition environment. Specifically, the camera device for acquiring the template image can be the same camera device as the camera device for acquiring the business image. Optionally, the camera device for acquiring the template image can also be a different camera device from the camera device for acquiring the business image. However, in order to ensure the accuracy of auxiliary positioning of the business object in the business image through the template object in the template image, the embodiment of the present application proposes that the camera device for acquiring the template image can have the same setting parameters as the camera device for acquiring the business image. For example, the distance from the template object to the camera device can be set to be consistent with the distance from the target object to the camera device, so as to ensure that the resolution of the acquired template image can be consistent with the resolution of the business image. In this way, the size of the template object displayed in the template image can be basically consistent with the size of the target object displayed in the business image (for example, there may be slight differences in the size of the object due to differences in the positions of the objects, but since only a rough positioning is performed, this can be ignored here). It can be understood that in some industrial scenarios, in order to improve the accuracy of product measurement of industrial products (for example, edge detection in product quality inspection scenarios), when the embodiment of the present application captures images of industrial products (such as target objects) to obtain business images, a camera device with less distortion and higher clarity can be used for image capture to obtain a more accurate and complete business image that contains the target object. After fitting the accurate edge curve of the target object, the industrial product can be detected more accurately based on the edge curve.

[0068] In some cases, the size of the template image and the business image may be different. Therefore, when determining the object detection area for assisting the positioning of the target object in the template image, the template image can be scaled to obtain the scaled template image. It should be understood that the size of the scaled template image is consistent with the size of the business image currently obtained for the target object. Then, the embodiment of the present application can quickly determine the object detection area for assisting the positioning of the target object based on the scaled template image, so that the area size of the object detection area determined in the scaled template image can be smaller than the area size of the entire business image, so that the area matching the object detection area can be found in this business image later.

[0069] Specifically, the embodiments of the present application can also obtain scaled template images of different sizes by scaling the template image, and then quickly locate template object areas of different sizes from the scaled template images of different sizes (the template object area here can be the area where the template object is preliminarily located in the corresponding scaled template image), so that subsequent matching can be performed in the business image based on the template object areas of different sizes to determine matching areas that match these template object areas of different sizes in the business image (the matching areas here refer to areas that match the template object areas of each different size found in the business object), so that the object detection area for assisting in positioning the target object can be accurately determined based on the maximum area similarity between the determined matching area and the template object area of ​​the corresponding size. In other words, the embodiment of the present application can use template object areas of different sizes to search for matching areas with the greatest regional similarity with template object areas of each size in the business image, and record the maximum regional similarity corresponding to the template object area of ​​each size, and then determine the target maximum regional similarity from the maximum regional similarity corresponding to the template object area of ​​each size, so as to determine the template object area corresponding to the target maximum regional similarity as the object detection area, and can determine the matching area with the target maximum regional similarity with the object detection area in the business image as the area that matches the object detection area. It is understandable that when scaling the template image, if the template image needs to be reduced, pixel sampling can be performed according to the reduced image size, and if the template image needs to be enlarged, pixel interpolation can be performed according to the enlarged image size.

[0070] Optionally, the template image and the business image may also include the same reference object, which can be used as an object required for reference when scaling the template image. In other words, when capturing the template image, the template object and the reference object are photographed at the same time, so that the captured template image can include both the template object and the reference object. Similarly, when capturing the business image, the target object and the reference object are photographed, so that the captured business image can also include the target object to be detected and the reference object used to scale the template image. In other words, the size of the reference object in the scaled template image can be consistent with the reference image in the business object. Specifically, when scaling the template image, the embodiment of the present application can make the area size of the reference object in the scaled template image consistent with the area size of the reference object in the business image, and then the area where the object detection frame of the located template object is located can be determined as the above-mentioned object detection area in the scaled template image.

[0071] Step S102: Search for an area in the business image that matches the object detection area, determine the found area as the initial positioning area of ​​the target object, and determine the pixel position transformation relationship based on the pixel positions of the pixel points in the object detection area and the pixel positions of the pixel points in the initial positioning area.

[0072] The initial positioning area refers to the area found in the service image that matches the object detection area. For example, the area matching the object detection area may specifically refer to the matching area with the greatest regional similarity to the object detection area found through regional pixel matching in the service image.

[0073] It can be understood that in the process of searching for an area matching the object detection area in the business image, a sliding window having a size consistent with the area size of the object detection area determined in the business image can be prioritized in accordance with the above-mentioned regional pixel matching method. For example, the embodiment of the present application can continuously slide the sliding window on the business image by the sliding window method in accordance with the above-mentioned regional pixel matching method, and calculate the regional similarity between the area where the sliding window is located and the object detection area when the sliding window slides to each position. Then, in the obtained regional similarity, the area where the sliding window corresponding to the maximum regional similarity is located can be determined as the area matching the object detection area.

[0074] Specifically, the specific process of searching for an area that matches the object detection area in a business image in an embodiment of the present application can be described as the following steps: the computer device can determine a sliding window for window sliding in the business image based on the area size of the object detection area, and then determine the window position of the sliding window in the business image as the area to be matched in the business image; the area size of the area to be matched here is consistent with the area size of the object detection area; further, the computer device can determine the area similarity between the area to be matched and the object detection area through the pixel value of each pixel point in the area to be matched and the pixel value of each pixel point in the object detection area; further, the computer device can determine the area that matches the object detection area in the business image based on the area similarity.

[0075] Among them, it can be understood that the sliding window refers to a window with the same area size as the object detection area that is predetermined when the window is sliding in the business image using the sliding window method. The size of the sliding window is consistent with the area size of the object detection area. In this way, the initial positioning area containing the complete target object can be preliminarily located in the process of sliding the window through the sliding frame. Among them, it can be understood that the area to be matched refers to the area corresponding to the window position where the sliding window is located. It can be understood that since the size of the sliding window is consistent with the area size of the object detection area, the area size of the area to be matched should also be consistent with the area size of the object detection area. It can be understood that the sliding window can slide to different window positions in the business image, and in the process of sliding the sliding window, the area corresponding to each window position to which the sliding window slides can be called the area to be matched.

[0076] It can be understood that the regional similarity refers to the similarity between the area to be matched and the object detection area calculated according to the above-mentioned regional pixel matching method. The regional pixel matching method here specifically refers to determining the regional similarity between the area to be matched and the object detection area based on the pixel value of each pixel point in the area to be matched and the pixel value of each pixel point in the object detection area. Specifically, when the area size of the object detection area is smaller than the area size of the business image, the different window positions to which the sliding window slides can be obtained in the business image by sliding the window. Therefore, the embodiment of the present application can also calculate the similarity between the area corresponding to each window position of the sliding window (i.e., each different area to be matched) and the object detection area during the sliding of the sliding window, and then the similarity between the area corresponding to each window position and the object detection area can be collectively referred to as the regional similarity between the area to be matched and the object detection area. For example, the calculated similarity between the area corresponding to window position 1 (i.e., the area to be matched) and the object detection area can be similarity 1, and the calculated similarity between the area corresponding to window position 2 (i.e., another area to be matched) and the object detection area can be similarity 2. Here, the calculated similarity 1 and similarity 2 can be collectively referred to as regional similarity.

[0077] It can be understood that the embodiments of the present application can also determine the area that matches the object detection area in the business image based on regional similarity. For example, the area to be matched that has the greatest regional similarity with the object detection area (for example, the area corresponding to the above-mentioned window position 1) can be determined as the area that matches the object detection area.

[0078] Specifically, the area to be matched here may include a first sliding window area and a second sliding window area; wherein the first sliding window area may be an area determined by sliding the sliding window to the first window position; similarly, the second sliding window area may be an area determined by sliding the sliding window to the second window position; the second window position here is the next window position of the first window position determined after sliding the sliding window according to the sliding step (in other words, when the window position corresponding to the sliding window is the first window position, the sliding window at the first window position may be slid according to the sliding step, and the window position of the sliding window after the window sliding may be referred to as the second window position. In other words, at this time, the second window position is the next window position of the first window position); then, at this time, the embodiment of the present application compares the pixel value of each pixel point in the area to be matched with the pixel value of each pixel point in the object detection area. The specific implementation method of determining the regional similarity between the to-be-matched area and the object detection area by comparing the pixel value of each pixel point in the first sliding window area with the pixel value of each pixel point in the object detection area can be described as the following steps: For example, specifically, the embodiment of the present application can determine the regional similarity between the first sliding window area and the object detection area through the pixel value of each pixel point in the first sliding window area and the pixel value of each pixel point in the object detection area, and determine the regional similarity between the first sliding window area and the object detection area as the first regional similarity; further, the embodiment of the present application can determine the regional similarity between the second sliding window area and the object detection area through the pixel value of each pixel point in the second sliding window area and the pixel value of each pixel point in the object detection area, and determine the regional similarity between the second sliding window area and the object detection area as the second regional similarity; then, the embodiment of the present application can determine the regional similarity between the to-be-matched area and the object detection area based on the first regional similarity and the second regional similarity.

[0079] It is understandable that the sliding window can slide on the business image according to a certain step size (also called sliding step size) until the business image is traversed and slid (also called traversal sliding end). The step size refers to the number of pixels that the sliding window needs to move each time it slides to the next window position. For example, the step size can be 1 pixel, 2 pixels, etc. Generally speaking, in order to improve the accuracy of the area determined to match the object detection area, the step size can be set to 1 pixel, and then the sliding window can be slid to all possible positions in the business image, so that the location of the target object can be matched more accurately.

[0080] Optionally, in some implementations, if it is necessary to further improve the efficiency of determining the area that matches the object detection area, the step size can be set to a larger number of pixels, thereby reducing the number of window positions involved in the regional similarity calculation. In this way, the number of times the regional similarity needs to be calculated can be reduced to a certain extent, thereby improving the efficiency of preliminarily locating the initial positioning area of ​​the target object in the business image through regional pixel matching. It is understandable that when the sliding window is slid in the business image, it can be slid in a certain sliding order. For example, the sliding order can be sliding from top to bottom, from left to right, etc. in the business image. The specific sliding direction of the sliding window will not be limited here. For example, when the sliding step is 1 pixel, the upper left corner point of the sliding window can be aligned with the upper left corner point of the business image (for example, overlap) in the business image to determine a sliding window with an area size consistent with the object detection area. Then, the sliding window can be traversed and slid in the business image from left to right in a sliding direction with a sliding step of 1 pixel until a left-to-right traversal sliding is completed in the sliding direction. The sliding window can then be slid down one pixel according to the sliding step, and the sliding window can be traversed and slid in the business image from left to right again with a sliding step of 1 pixel until another left-to-right traversal sliding is completed in the sliding direction. Similarly, the sliding window is traversed and slid in the business image according to the above sliding order from left to right until the entire business image is traversed and slid, and the traversal and sliding of the business image is terminated. For example, when the lower right corner point of the sliding window is aligned with the lower right corner point of the business image, it can be determined that the entire business image has been slid (i.e., it is determined that the traversal and sliding of the entire business image has been completed), and thus the traversal and sliding of the business image can be terminated. Among them, traversing and sliding the business image means that when the sliding window slides according to the sliding step size and sliding order, the sliding window slides to the last window position, and the lower right corner point of the sliding window corresponding to the last window position is aligned with the lower right corner point of the business image (e.g., overlapped).

[0081] The first sliding window area is different from the second sliding window area. The first sliding window area is the area to be matched determined when the sliding window slides to the first window position, and the second sliding window area is the area to be matched determined when the sliding window slides to the second window position. The first window position is different from the second window position. It is understandable that, in one implementation, the first window position can be specifically the first window position (for example, the first window position) among the window positions to which the sliding window slides in the business image, and the second sliding window area can be specifically the next window position of the first window position among the window positions to which the sliding window slides (for example, the next window position of the first window position). Optionally, in another implementation, the first window position can be specifically a non-first window position among the window positions to which the sliding window slides in the business image (for example, the next window position of the aforementioned first window position), and the second sliding window area can be specifically the next window position of the non-first window position among the window positions to which the sliding window slides. The specific window positions of the first window position and the second window position are not limited here.

[0082] The first region similarity refers to the region similarity between the first sliding window region and the object detection region, and the second region similarity refers to the region similarity between the second sliding window region and the object detection region.

[0083] It can be understood that in one implementable method, if when the sliding window is slid to the second window position, it is determined that the sliding window has completed sliding in the business image, then the specific steps of the embodiment of the present application in determining the regional similarity between the area to be matched and the object detection area based on the first area similarity and the second area similarity can be described as: determining the first area similarity and the second area similarity as the regional similarity between the area to be matched and the object detection area.

[0084] Optionally, if, when sliding the sliding window to the second window position, it is determined that the sliding window has not completely traversed the business image, the specific process of determining the regional similarity between the area to be matched and the object detection area based on the first region similarity and the second region similarity in the embodiment of the present application can also be described as: sliding the sliding window from the second window position to the next window position of the second window position according to the sliding step size, and determining the area corresponding to the next window position of the second window position to which the sliding window is currently slid as the third sliding window area, so that the regional similarity between the third sliding window area and the object detection area can be further determined by comparing the pixel value of each pixel point in the third sliding window area with the pixel value of each pixel point in the object detection area, so as to determine the regional similarity between the third sliding window area and the object detection area as the third region similarity, thereby determining the regional similarity between the area to be matched and the object detection area based on the first region similarity, the second region similarity, and the third region similarity.

[0085] For example, the embodiment of the present application can use the current window position of the sliding window (i.e., the second window position) as the new first window position, and then slide the sliding window from the new first window position (i.e., the second window position) to the new second window position (i.e., the new second window position is essentially the next window position of the second window position, for example, the third window position) according to the sliding step size. To facilitate distinction, the embodiment of the present application can determine the area corresponding to the new second window position (i.e., the next window position of the second window position, for example, the third window position) to which the sliding window currently slides in the business image as the third sliding window area; in this way, the embodiment of the present application can further determine the regional similarity between the third sliding window area and the object detection area by comparing the pixel value of each pixel point in the third sliding window area with the pixel value of each pixel point in the object detection area, and then determine the regional similarity between the third sliding window area and the object detection area as the third regional similarity. Then, the embodiment of the present application can determine the regional similarity between the to-be-matched area and the object detection area based on the first regional similarity, the second regional similarity, and the third regional similarity.

[0086] Optionally, in another implementation manner, for any two sliding window areas associated with the sliding window (i.e., the aforementioned first sliding window area and the second sliding window area), since the area similarity between the first sliding window area and the object detection area is the first area similarity, and the area similarity between the second sliding window area and the object detection area is the second area similarity, the specific process of determining the area similarity between the to-be-matched area and the object detection area based on the first area similarity, the second area similarity, and the third area similarity in the embodiment of the present application can be described as follows: first, the first area similarity and the second area similarity are compared to obtain a first comparison result; based on the first comparison result, the maximum area similarity determined between the first area similarity and the second area similarity is used as a candidate target area similarity; then, the candidate target area similarity can be compared with the third area similarity to obtain a second comparison result; based on the second comparison result, the maximum area similarity determined between the candidate target area similarity and the third area similarity is used as the target area similarity; then, the to-be-matched area corresponding to the target area similarity can be determined as the area matching the object detection area.

[0087] It should be understood that in the embodiment of the present application, when a number of to-be-matched areas are determined in a business image by means of a sliding window, the regional similarities of any two adjacent to-be-matched areas can also be compared by means of pairwise comparison, so that the regional similarity with the maximum value among the regional similarities of any two adjacent to-be-matched areas can be used as the candidate target region similarity, and then the candidate target region similarity with the maximum value can be further used as the target region similarity from these determined candidate target region similarities.

[0088] It can be understood that after calculating the regional similarity corresponding to the second sliding window area, the sliding window can be slid to the next window position according to the sliding step, and the regional similarity corresponding to the area to be matched at the next window position can be calculated until the sliding window finishes traversing the business image. In this way, during the sliding of the sliding window, the regional similarity between the area to be matched and the object detection area corresponding to each window position to which the sliding window slides can be recorded, so that among these regional similarities, the final regional similarity with the maximum value can be used as the target regional similarity, and then the area corresponding to the target regional similarity can be determined as the area matching the object detection area in the business image.

[0089] Specifically, the embodiment of the present application can determine the target area similarity based on the first area similarity and the second area similarity, where the target area similarity is the area similarity with the maximum value determined in the first area similarity and the second area similarity. In other words, the target area similarity can be the maximum area similarity in the first area similarity and the second area similarity; in this way, the embodiment of the present application can determine the area that matches the object detection area in the business image based on the area to be matched corresponding to the target area similarity.

[0090] The target region similarity is the maximum region similarity between the first region similarity and the second region similarity. In other words, the region that matches the object detection region is the target matching region determined from the various to-be-matched regions obtained during the sliding window sliding process. The target matching region is the region with the maximum region similarity ultimately determined from the recorded region similarities of the various to-be-matched regions.

[0091] It is understood that when the regional similarities include more regional similarities (e.g., more than two regional similarities), the regional similarity with the maximum value among all the regional similarities of the to-be-matched regions can be used to determine the target regional similarity. For example, if the regional similarities of all the to-be-matched regions include the first regional similarity, the second regional similarity, and the third regional similarity, the maximum regional similarity determined among the first regional similarity, the second regional similarity, and the third regional similarity can be determined as the target regional similarity.

[0092] For example, please refer to Figure 5, which is a schematic diagram of a window sliding scenario provided by an embodiment of the present application. As shown in Figure 5, when the sliding window is at window position 1, the window position corresponding to window position 1 can be determined as the first window position, and the area where the window position 1 is located in the business image is the first window sliding area, and then the area similarity 1 corresponding to the window position 1 can be determined (that is, the area similarity between the area to be matched where the window position 1 is located and the object detection area can be calculated); further, the embodiment of the present application can slide the sliding window from window position 1 to the next window position (that is, window position 2 shown in Figure 5) in the sliding direction (for example, a sliding order from left to right) according to a preset sliding step size (for example, a step size of 1 pixel).

[0093] For example, as shown in Figure 5, after the sliding window is slid to the right by one pixel, the current window position of the sliding window can be displayed on the window position 2 shown in Figure 5, so the window position 2 can be determined as the above-mentioned second window position (that is, the next window position of the first window position). It should be understood that at this time, the area where the window position 2 is located is the second window sliding area, and then the regional similarity 2 between the area to be matched (that is, the second window sliding area) corresponding to the window position 2 and the object detection area can be determined (that is, calculated) according to the above-mentioned regional pixel matching method.

[0094] Furthermore, as shown in FIG5 , the sliding window can also be slid from window position 2 to the next window position (i.e., window position 3 shown in FIG5 ). For example, as shown in FIG5 , after the sliding window is further slid to the right by one pixel, the current window position of the sliding window can be displayed at window position 3 shown in FIG5 . At this time, the area where the window position 3 is located is the third window sliding area, and then the regional similarity 3 between the to-be-matched area (i.e., the third window sliding area) corresponding to the window position 3 and the object detection area can be determined (i.e., calculated) in accordance with the above-mentioned regional pixel matching method. Further, by analogy, until the sliding window is traversed and slid in the business image (i.e., as shown in FIG5 , until the current window position of the sliding window is displayed on the window position n), the regional similarity (such as regional similarity 1, regional similarity 2, regional similarity 3, etc.) of the area to be matched corresponding to each window position can be recorded during the entire sliding process of the sliding window, so that when the maximum regional similarity among the regional similarities (such as regional similarity 1, regional similarity 2, regional similarity 3, etc.) of the area to be matched corresponding to each window position is determined as the target area similarity, the area to be matched corresponding to the target area similarity is determined as the area that matches the object detection area. In other words, the embodiment of the present application can determine the area to be matched corresponding to the maximum regional similarity as the area that matches the object detection area in the area to be matched where each window position is located.

[0095] For ease of understanding, the embodiment of the present application takes the number of areas to be matched as one as an example, and explains the specific process of determining the regional similarity between the area to be matched and the object detection area according to the above-mentioned regional pixel matching method. That is, for any area to be matched, the embodiment of the present application can specifically determine the regional similarity between the area to be matched and the object detection area through the pixel value of each pixel point in the area to be matched and the pixel value of each pixel point in the object detection area.

[0096] For example, an embodiment of the present application can determine the regional similarity between the area to be matched and the object detection area based on the pixel difference between each pixel point in the area to be matched and the pixel point at the same pixel coordinate in the object detection area. In other words, an embodiment of the present application can calculate the average value of the pixel difference between two pixel points at the same pixel coordinate based on the pixel difference between each pixel point in the area to be matched and the pixel point at the same pixel coordinate in the object detection area, and then determine the regional similarity between the area to be matched and the object detection area based on the calculated average value of the pixel difference. It can be understood that the pixel difference can refer to the difference between the pixel values ​​of two pixel points at the same pixel coordinate. It should be understood that the greater the average value of the pixel difference between the pixel points at each pixel coordinate, the smaller the regional similarity, and conversely, the smaller the average value of the pixel difference between the pixel points at each pixel coordinate, the greater the regional similarity.

[0097] The pixel coordinates here refer to the pixel position corresponding to the row and column where the pixel point is located in an area (such as the area to be matched or the object detection area). For example, in the area to be matched, the pixel coordinates of the pixel point at the 1st row and 2nd column can be expressed as (1, 2).

[0098] Optionally, in one or more possible implementations, for any area to be matched (such as the first sliding window area and the second sliding window area mentioned above), the regional similarity between the area to be matched and the object detection area is determined by comparing the pixel value of each pixel in the area to be matched and the pixel value of each pixel in the object detection area. Specifically, the method may include: the computer device may determine the pixel mean difference corresponding to each pixel in the area to be matched based on the pixel difference between the pixel value of each pixel in the area to be matched and the pixel mean of the area to be matched (for example, the average value calculated by the pixel values ​​of each pixel in the area to be matched); further, the computer device may determine the pixel mean difference corresponding to each pixel in the object detection area based on the pixel difference between the pixel value of each pixel in the object detection area and the pixel mean of the object detection area (for example, another average value calculated by the pixel values ​​of each pixel in the object detection area); further, the computer device may determine the pixel mean difference corresponding to each pixel in the object detection area based on the pixel difference between the pixel mean difference corresponding to each pixel in the area to be matched and the pixel mean difference corresponding to the pixels with the same pixel coordinates in the object detection area.

[0099] The pixel mean (also called pixel average) of the area to be matched refers to the average value obtained by averaging the pixel values ​​of all pixels in the area to be matched. Similarly, the pixel mean (also called pixel average) of the object detection area refers to another average value obtained by averaging the pixel values ​​of all pixels in the object detection area. In addition, the pixel mean difference corresponding to each pixel here refers to the difference between the pixel value of each pixel in a certain area (such as the object detection area or the area to be matched) and the pixel mean of that area.

[0100] For example, in an embodiment of the present application, in the process of determining the regional similarity between the to-be-matched area and the object detection area based on the pixel mean difference corresponding to each pixel point in the to-be-matched area and the pixel mean difference corresponding to the pixel points at the same pixel coordinates in the object detection area, the pixel standard deviation (or variance) associated with the pixel values ​​of the pixel points in the to-be-matched area can be first obtained, and then the pixel standard deviation (or variance) associated with the pixel values ​​of the pixel points in the object detection area can be obtained. Then, based on the pixel mean difference corresponding to each pixel point in the to-be-matched area, the pixel mean difference corresponding to the pixel points at the same pixel coordinates in the object detection area, the pixel standard deviation (or variance) associated with the pixel values ​​of the pixel points in the to-be-matched area, and the pixel standard deviation (or variance) associated with the pixel values ​​of the pixel points in the object detection area, the regional similarity between the to-be-matched area and the object detection area can be determined. Wherein, the pixel standard deviation (or variance) refers to the variance obtained by calculating the standard deviation of the pixel values ​​of the pixel points in a certain area (e.g., the to-be-matched area or the object detection area). For example, an embodiment of the present application can perform standard deviation calculation on the pixel values ​​of each pixel point in the to-be-matched area to calculate the variance of the pixel points in the to-be-matched area. Similarly, an embodiment of the present application can also perform standard deviation calculation on the pixel values ​​of each pixel point in the object detection area to calculate the variance of the pixel points in the object detection area.

[0101] For example, the regional similarity between the to-be-matched region and the object detection region can be determined by using the above-mentioned regional pixel matching method, which can refer to the following formula (1):

[0102] In the formula (1), Sim can be used to represent the regional similarity between the to-be-matched region (e.g., the to-be-matched region p) and the object detection region (e.g., the object detection region q). In the formula (1), p can be used to represent the to-be-matched region, and similarly, q can be used to represent the object detection region, wherein p(x, y) is used to represent the pixel value of a pixel at a certain pixel coordinate (e.g., the pixel position corresponding to the pixel point on the xth row and yth column) in the to-be-matched region (e.g., the to-be-matched region p), and q(x, y) is used to represent the pixel value of another pixel at the same pixel coordinate (e.g., the pixel position corresponding to another pixel point on the xth row and yth column) in the object detection region (e.g., the object detection region q), wherein μ f Used to represent the average pixel value of the area to be matched, μ t It is used to represent the average pixel value of the object detection area. Similarly, σ f It refers to the pixel standard deviation of the area to be matched obtained by calculating the standard deviation of the pixel values ​​of the pixels in the area to be matched, and σ t It refers to the pixel standard deviation of the object detection area obtained by calculating the standard deviation of the pixel values ​​of the pixel points in the object detection area. In addition, in formula (1), m -1 is a hyperparameter that can be used as a scaling factor.

[0103] For ease of understanding, here we take the example of the number of areas to be matched being 1, and this area to be matched can be the above-mentioned first sliding window area. At this time, the first sliding window area includes the i-th pixel point, and the object detection area includes the j-th pixel point; i and j are consistent positive integers, and the pixel position of the i-th pixel point in the first sliding window area is consistent with the pixel position of the j-th pixel point in the object detection area; based on this, the computer device determines the specific process of the regional similarity between the first sliding window area and the object detection area through the pixel value of each pixel point in the first sliding window area and the pixel value of each pixel point in the object detection area. It can be described as: the computer device can calculate the average value of the pixel values ​​of the pixels in the first sliding window area and determine it as the first pixel value of the first sliding window area. mean; further, the computer device may determine the average value obtained by performing mean calculation on the pixel values ​​of the pixel points in the object detection area as the second pixel mean of the object detection area; further, the computer device may determine the difference between the pixel value of the i-th pixel point in the first sliding window area and the first pixel mean as the first difference corresponding to the i-th pixel point, and determine the difference between the pixel value of the j-th pixel point in the object detection area and the second pixel mean as the second difference corresponding to the j-th pixel point; further, the computer device may determine the regional similarity between the first sliding window area and the object detection area based on the first difference corresponding to the i-th pixel point and the second difference corresponding to the j-th pixel point.

[0104] The i-th pixel point may be any pixel point in the first sliding window area. For example, if the size of the first sliding window area (i.e., the area size) is W×E (i.e., the first sliding window area may specifically include pixels with a number of W rows×E columns), based on the above formula (1), the i-th pixel point may be the i-th pixel point on the x-th row and y-th column in the above-mentioned area to be matched p. For example, the i-th pixel point may be the pixel point on the 1st row and 1st column, the pixel point on the 1st row and 2nd column, ..., the pixel point on the Wth row and Eth column, etc. in the first sliding window area. Similarly, the jth pixel point can be any pixel point in the object detection area. For example, the size of the object detection area (i.e., the area size) is W×E (i.e., the object detection area can specifically include pixels with a number of W rows×E columns). Based on the above formula (1), the jth pixel point can be the jth (here j=i) pixel point on the xth row and yth column in the above object detection area q. For example, the jth pixel point can be the pixel point in the 1st row and 1st column, the pixel point in the 1st row and 2nd column, ..., the pixel point in the Wth row and Eth column, etc. in the object detection area.

[0105] It can be understood that the first pixel mean refers to the average value obtained by averaging the pixel values ​​of the pixels in the first sliding window area. The second pixel mean refers to the average value obtained by averaging the pixel values ​​of the pixels in the object detection area. In addition, the first difference refers to the difference between the pixel value of the i-th pixel in the first sliding window area and the first pixel mean. Similarly, the second difference refers to the difference between the pixel value of the j-th pixel in the object detection area and the second pixel mean.

[0106] Among them, the specific process of determining the regional similarity between the first sliding window area and the object detection area based on the first difference corresponding to the i-th pixel point and the second difference corresponding to the j-th pixel point in the embodiment of the present application can be calculated with reference to the above formula (1). For example, the pixel value of the i-th pixel point in the first sliding window area can be p(x, y) in formula (1). Similarly, the pixel coordinates of the j-th pixel point in the object detection area are q(x, y) in formula (1). In addition, (p(x, y)-μ in formula (1) f ) can be the first difference value corresponding to the i-th pixel point. Similarly, (q(x,y)-μ t ) can be the second difference value corresponding to the j-th pixel point.

[0107] It can be understood that the specific implementation method of the embodiment of the present application in which the regional similarity between the second sliding window area and the object detection area is determined by the regional pixel matching method indicated by the above formula (1) can refer to the above description of determining the regional similarity between the first sliding window area and the object detection area by the regional pixel matching method, and will not be repeated here.

[0108] For example, specifically, the second sliding window area includes the e-th pixel point, and the object detection area includes the j-th pixel point; e and j are consistent positive integers, and the pixel position of the e-th pixel point in the second sliding window area is consistent with the pixel position of the j-th pixel point in the object detection area; then, the computer device determines the specific process of the regional similarity between the second sliding window area and the object detection area through the pixel value of each pixel point in the second sliding window area and the pixel value of each pixel point in the object detection area, which can be described as: the computer device can determine the average value obtained by performing mean calculation on the pixel values ​​of the pixels in the second sliding window area as the second sliding window area. The computer device may determine the third pixel mean of the sliding window area; and determine the average value obtained by averaging the pixel values ​​of the pixels in the object detection area as the second pixel mean; then, the computer device may determine the difference between the pixel value of the e-th pixel in the second sliding window area and the third pixel mean as the third difference corresponding to the e-th pixel, and determine the difference between the pixel value of the j-th pixel in the object detection area and the second pixel mean as the second difference corresponding to the j-th pixel; further, the computer device may determine the regional similarity between the second sliding window area and the object detection area based on the third difference corresponding to the e-th pixel and the second difference corresponding to the j-th pixel. The third pixel mean refers to the average value obtained by averaging the pixel values ​​of the pixels in the second sliding window area. The third difference refers to the difference between the pixel value of the e-th pixel in the second sliding window area and the third pixel mean.

[0109] It can be understood that in an embodiment of the present application, the pixel position transformation relationship can be used to transform the pixel points in the initial positioning area into the object detection area, that is, the pixel position transformation relationship refers to a first transformation relationship constructed based on the pixel position of the pixel point in the initial positioning area and the pixel position of the corresponding pixel point in the object detection area. The first transformation relationship can be a forward transformation relationship for transforming from the initial positioning area to the object detection area. In other words, the pixel position transformation relationship is used to characterize which transformation relationship (for example, a forward transformation relationship) can be used to transform (for example, forward transformation) the pixel position of the pixel point in the initial positioning area into the pixel position of the pixel point mapped in the object detection area, and then the pixel point mapped in the business image can be determined by the currently obtained pixel position transformation relationship and the pixel position of the pixel point in the object detection area. For example, the pixel position of the pixel point in the object detection area can be reversely transformed by the currently obtained pixel position transformation relationship (for example, a reverse transformation relationship further determined by the forward transformation relationship) to transform (for example, reverse transformation) the pixel position of the pixel point in the object detection area into the pixel point mapped in the initial positioning area. In this way, after a local area that needs to be focused on is pre-marked in the object detection area of ​​the template image (for example, the local reference area 223a shown in Figure 2 above), the pixel position transformation relationship currently constructed can be used to more accurately and efficiently determine the local mapping area to be used for foreground recognition in the business image, and then the local mapping area determined in the business image can be used as the local area of ​​the target object determined in the business image that needs to be focused on, and then in the process of foreground area recognition through the local mapping area, the accuracy and efficiency of foreground recognition can be improved.

[0110] Specifically, the embodiment of the present application can determine the pixel position transformation relationship based on the pixel position of the pixel points in the object detection area and the pixel position of the pixel points in the initial positioning area, that is, it can construct a pixel position transformation relationship for transforming from the initial positioning area to the object detection area.

[0111] Among them, the specific process of the computer device constructing the pixel position transformation relationship can be described as: the computer device can determine K key pixel points from the initial positioning area; K is a positive integer greater than 1; further, the computer device can determine K associated pixel points corresponding to the K key pixel points in the object detection area; it should be understood that one key pixel point here can correspond to one associated pixel point; further, the computer device can construct K pixel pairs based on the K key pixel points and the K associated pixel points; one pixel pair here includes a key pixel point and a corresponding associated pixel point; further, the computer device can determine the pixel positions of the K key pixel points and the pixel positions of the K associated pixel points in the target coordinate system, and then can calculate the pixel position transformation matrix used to characterize the pixel position transformation relationship based on the pixel positions of the key pixel points and the pixel positions of the associated pixel points included in each pixel pair in the K pixel pairs.

[0112] Among them, the key pixel points can be the pixel points determined in the initial positioning area for calculating the pixel position transformation relationship. For example, the key pixel points here can be all the pixel points in the initial positioning area or some of the pixel points selected in the initial positioning area. Similarly, the associated pixel points can be the pixel points in the object detection area that have the same pixel coordinates as the key pixel points. Generally speaking, in order to more conveniently determine the associated pixel points corresponding to the key pixel points, when the embodiment of the present application selects some pixel points from the initial positioning area as key pixel points, the pixel points at each corner position and / or center position in the initial positioning area can be used as key pixel points. For example, when the initial positioning area is a rectangular area, the corner points here can specifically refer to the four vertices of this rectangular area.

[0113] For another example, in one or more possible implementations, if the key pixel point is a pixel point at the pixel coordinates indicated by the 1st row and 1st column selected in the initial positioning area, then the associated pixel point corresponding to the key pixel point may be a pixel point at the same pixel coordinates specified by the 1st row and 1st column selected in the object detection area. For another example, if the key pixel point is a pixel point at the pixel coordinates indicated by the 5th row and 5th column selected in the initial positioning area, then the associated pixel point corresponding to the key pixel point may be a pixel point at the same pixel coordinates specified by the 5th row and 5th column selected in the object detection area. It should be understood that the embodiments of the present application will not limit the number of key pixel points selected in the initial positioning area (i.e., the value of K).

[0114] It is understandable that a key pixel can correspond to a related pixel, and a pixel pair can be constructed based on a key pixel and a corresponding related pixel. Then, for K key pixels, K pixel pairs can be constructed based on these K key pixels and the K related pixels corresponding to these K key pixels. In other words, a pixel pair is composed of a key pixel selected in the initial positioning area and a related pixel with the same pixel coordinates determined in the object detection area.

[0115] The pixel position here refers to the position of the pixel point in the target coordinate system. It is understandable that when determining the pixel position transformation matrix, it is necessary to transform the pixel points in the initial positioning area and the pixel points in the object detection area into the same coordinate system (i.e., the target coordinate system), so that the pixel position transformation relationship for transforming from the initial positioning area to the object detection area can be determined in the same coordinate system (i.e., the target coordinate system).

[0116] Among them, the target coordinate system can be a two-dimensional coordinate system, that is, the two-dimensional coordinate system can specifically include an x-axis (also called a first coordinate axis) and a y-axis (also called a second coordinate axis), and the origin of the target coordinate system can be any position in the plane where the corresponding image is located. For example, in the process of constructing the target coordinate system, the embodiment of the present application can determine the upper left corner point of the business image (or template image) as the origin, and then the two sides parallel to the length and width of the business image (or object detection area) can be determined as the x-axis and y-axis respectively. It can be understood that the target coordinate system can be specifically an image coordinate system, which uses physical units (such as millimeters) to represent the position of the pixel point in the image. For example, the pixel position of a pixel point can be expressed as (10,11), which means that in the target coordinate system, the value of the pixel point on the x-axis can be 10, and the value on the y-axis can be 11.

[0117] It should be understood that the target coordinate system here can be a coordinate system constructed for determining the pixel positions of pixel points in the initial positioning area and the object detection area, and then based on the target coordinate system, the pixel points in the initial positioning area and the pixel points in the object detection area can be converted to the same coordinate system, so that the pixel positions of the pixel points in the initial positioning area in the target coordinate system can be further determined in the same coordinate system, and the pixel positions of the pixel points in the object detection area in the target coordinate system can be determined.

[0118] It is understood that in the embodiment of the present application, at least two pixel pairs are required in the process of calculating the pixel position transformation matrix used to represent the pixel position transformation relationship. And the pixel position transformation matrix can be expressed as the following formula (2):

[0119] In formula (2), M0 can be a pixel position transformation matrix used to characterize the pixel position transformation relationship. x Indicates the distance required to move on the x-axis to transform the pixel point in the initial positioning area to the pixel point with the same pixel coordinate in the object detection area in the target coordinate system; similarly, the matrix element t y Indicates the distance that needs to be moved on the y-axis to transform the pixel point in the initial positioning area to the pixel point at the same pixel coordinate in the object detection area in the target coordinate system. Based on this, the embodiment of the application can also be obtained by multiplying the pixel position of the pixel point in the object detection area by the inverse matrix (i.e., M0) of the pixel position transformation matrix (i.e., M1). inv =M0 -1 ), and then determine the pixel point corresponding to the calculated mapping pixel position as the mapping pixel point in the initial positioning area (that is, the mapping pixel point in the initial positioning area is the pixel point obtained by inversely transforming the pixel point in the object detection area). In other words, the embodiment of the present application can quickly determine the mapping pixel point of the pixel point in the object detection area in the initial positioning area through the pixel position transformation matrix (specifically, it can be the inverse matrix of the pixel position transformation matrix). It should be understood that in the embodiment of the present application, the inverse matrix (that is, M inv ) means that the inverse transformation operation opposite to the original matrix (ie, M0) needs to be performed, which means that the embodiment of the present application can be achieved by the inverse matrix (ie, M0) of the pixel position transformation matrix (ie, inv =M0 -1 ), and reversely transform the pixels in the object detection area to the initial positioning area. In this embodiment, the pixel position transformation matrix (ie, M0) for forward transformation and the inverse matrix (ie, M0) for reverse transformation can be used as the matrix. inv =M0 -1 ) are collectively referred to as the constructed pixel position transformation relationship. Among them, the pixel position transformation relationship corresponding to the pixel position transformation matrix (ie, M0) can be the above-mentioned forward transformation relationship, and the inverse matrix (ie, M inv =M0 -1 ) can be the above-mentioned inverse transformation relationship.

[0120] Step S103: When the local reference area is determined from the template image, a local mapping area corresponding to the local reference area is determined in the service image based on the pixel position transformation relationship, and the local mapping area is used as the target positioning area of ​​the target object.

[0121] The local reference area may be a pre-set area in the template image for use as a reference in determining the local area to be processed in the business image. It is understood that the local reference area may be an area that is pre-marked in the template image by a user (e.g., a quality inspector) based on actual needs in an industrial scenario as a focus area, that is, the local reference area refers to the local area marked in the template image where edge curve fitting is required.

[0122] Among them, the local mapping area refers to the area mapped in the business image by reversely transforming the local reference area in the template image. In an embodiment of the present application, the local mapping area mapped in the business image can be collectively referred to as the target positioning area to be subjected to edge curve fitting, that is, the target positioning area refers to the area subsequently used for edge curve fitting, which means that the target positioning area involved in the embodiment of the present application can be determined based on the inverse matrix corresponding to the pixel position transformation relationship, after reversely transforming the pixel points in the local mapping area, the pixel points in the local reference area marked by the template image are reversely mapped to the mapping pixel points in the business image. It can be understood that determining the local reference area from the template image can specifically be determining (for example, identifying and locating) a pre-marked local reference area from the object detection area in the template image.

[0123] It can be understood that, in one implementable method, if the template image is scaled in advance when determining the object detection area, then when determining the local reference area here, the local reference area is still determined from the template image that has been scaled using the same scaling method.

[0124] Specifically, the computer device can determine (for example, select) P reference pixels in the local reference area; P is a positive integer greater than 1; further, the computer device can determine the pixel positions of the P reference pixels in the target coordinate system; further, the computer device can determine the P mapping pixels corresponding to the P reference pixels in the business image based on the pixel position transformation relationship (for example, the inverse matrix of the above-mentioned pixel position transformation matrix) and the pixel positions of the P reference pixels; wherein, one reference pixel corresponds to one mapping pixel; then, the computer device can determine the local mapping area corresponding to the local reference area in the business image based on the P mapping pixels. For example, in an embodiment of the present application, the area composed of these P mapping pixels can be used as the local mapping area corresponding to the local reference area determined in the business image. In other words, in an embodiment of the present application, the local mapping area refers to the area obtained after pixel positions are transformed on the pixels (for example, P reference pixels) in the local reference area based on the pixel position transformation relationship.

[0125] The reference pixel point may be a reference point selected in the local reference area for determining the local mapping area. Similarly, the mapping pixel point may be a pixel point determined in the business image and mapped by the corresponding reference pixel point. For example, the pixel position transformation relationship may be expressed as performing an inverse transformation on the pixel position transformation matrix (specifically, performing an inverse matrix transformation process) to obtain an inverse matrix of the pixel position transformation matrix (for example, the above-mentioned M inv ), and then the pixel position of the reference pixel point can be multiplied on the left by the inverse matrix of the pixel position transformation matrix to achieve accurate acquisition of the mapping pixel position corresponding to the reference pixel point through pixel position transformation, so that the pixel point corresponding to the mapping pixel position in the business image can be determined as the mapping pixel point, and one reference pixel point can correspond to one mapping pixel point.

[0126] It should be understood that the number of reference pixel points should be greater than 1 (that is, P is a positive integer greater than 1). In other words, the embodiment of the present application can ultimately locate the local mapping area that needs to focus on the local area of ​​the target object through the mapping pixel points corresponding to multiple reference pixel points.

[0127] For example, to facilitate understanding, here we take the P reference pixels as the four corner points of the local reference area as an example to illustrate the specific process of transforming the pixels in the pre-marked local reference area in the template image to the business image through the above pixel position transformation relationship. It should be understood that the embodiment of the present application can be achieved by the inverse matrix of the pixel position transformation matrix (for example, the above M inv ), the pixel positions of the four corner points selected from the local reference area are inversely transformed to transform the pixel points in the local reference area to the business image, so that the mapped pixel points corresponding to the four corner points (i.e., four mapped pixel points) can be obtained in the business image. Then, a rectangular area can be quickly determined in the business image based on the mapped pixel points corresponding to the four corner points (i.e., four mapped pixel points). At this time, the determined rectangular area is the local mapping area corresponding to the local reference area. It should be understood that in the embodiment of the present application, the area size of the local mapping area is consistent with the area size of the local reference area.

[0128] Among them, it can be understood that, in an embodiment of the present application, the target positioning area may refer to an area determined based on the local mapping area and used to perform curve fitting on the edge curve of the target object. Further, please refer to Figure 6, which is a schematic diagram of the effect of regional mapping provided by an embodiment of the present application. The coordinate system formed by the coordinate axes (for example, the x-axis and the y-axis) shown in Figure 6 is the above-mentioned target coordinate system. Under this target coordinate system, the area corresponding to the mark display box pre-marked by the quality inspector in the template image 601a can be the local reference area 611a shown in Figure 6. As shown in Figure 6, the computer device can select four vertices (that is, the above-mentioned 4 corner points) in the local reference area 611a as the above-mentioned P (for example, P=4) reference pixel points. For example, these 4 reference pixel points can specifically be the 4 reference pixel points 612a shown in Figure 6. Furthermore, the computer device can perform an inverse transformation on the four reference pixel points 612a based on the pixel position transformation relationship (e.g., the inverse matrix of the above-mentioned pixel position transformation matrix) to determine (i.e., transform) the mapping pixel position 613a corresponding to each reference pixel point 612a in the template image 601a. ​​It should be understood that the four mapping pixel point positions 613a obtained in the template image 601a can be shown in Figure 6. In this target coordinate system, each reference pixel point in the local reference area 611a can be calculated to obtain a corresponding mapping pixel position. Furthermore, the computer device can determine four mapping pixel points 621a in the business image 602a based on the four mapping pixel positions 613a, and can then determine the local mapping area 622a shown in Figure 6 in the business image 602a based on the area formed by these four mapping pixel points 621a. Among them, in the target coordinate system, a mapping pixel position 613a can be used to determine the pixel points on the same pixel coordinates in the business image 602a as a mapping pixel point to obtain the four mapping pixel points 621a shown in Figure 6, and then a local mapping area 622a can be determined based on the four mapping pixel points 621a. Then, the computer device can determine the local mapping area 622a as the target positioning area determined in the business image 602a that needs to focus on the local area of ​​the target object.

[0129] Step S104 : identifying a foreground area corresponding to the target object in the target positioning image corresponding to the target positioning area, performing edge curve fitting processing on the pixels in the foreground area, and obtaining an edge curve of the target object.

[0130] It can be understood that the target positioning image here refers to an image determined based on the target positioning area. For example, in one implementable manner, the computer device can crop the target positioning area in the business image to obtain the target positioning image.

[0131] It can be understood that the foreground area refers to the part with stronger visibility and significance in the target positioning image. Here, the foreground area can be the area where the target object is located that is identified and segmented from the target positioning image (for example, the local area of ​​the target object that needs to be focused on as mentioned above).

[0132] It is understandable that when the embodiment of the present application identifies the foreground area corresponding to the target from the target positioning image, it can be identified by a semantic segmentation network, a traditional threshold segmentation network, or it can be identified by using other deep learning methods such as instance segmentation, panoramic segmentation, etc., which are not limited here. It is understandable that there may be certain factors in the local area that will affect the extraction of the regional edge straight line, for example, features, shapes, and grayscale may change due to changes in the shooting background (such as a machine), the object itself (such as an industrial device), and the light source. Therefore, in order to avoid the influence of these factors existing in the local area on the extraction of the regional edge straight line, the embodiment of the present application proposes that the foreground area of ​​the local area (for example, the target positioning image corresponding to the above-mentioned target positioning area) can be identified by deep learning to tap into the deeper features existing in the target positioning image, rather than simply determining the foreground area based on the continuity of the grayscale value, thereby improving the accuracy of the foreground area identification, and then improving the accuracy of the fitted edge curve.

[0133] Specifically, the specific process of a computer device identifying the foreground area corresponding to the target object in the target positioning image corresponding to the target positioning area can be described as follows: the computer device can obtain a target foreground recognition network for performing foreground area recognition; further, the computer device can input the target positioning image corresponding to the target positioning area into the target foreground recognition network, and the target foreground recognition network performs foreground area recognition processing on the target positioning image to obtain the foreground probability of each pixel in the target positioning image; further, the computer device can determine the foreground area corresponding to the target object based on the foreground probability of each pixel in the target positioning image. It can be understood that the foreground probability here refers to the probability that each pixel identified in the target positioning image belongs to the foreground area. The larger the value of the foreground probability here, the greater the possibility that the pixel corresponding to the foreground probability belongs to the foreground area. Conversely, the smaller the value of the foreground probability, the smaller the possibility that the pixel corresponding to the foreground probability belongs to the foreground area.

[0134] Among them, the target foreground recognition network refers to a network used to identify the foreground area of ​​the target positioning image. The target foreground recognition network can be a neural network, for example, it can be the above-mentioned semantic segmentation network, the traditional threshold segmentation network, the instance segmentation network, the panoramic segmentation network, etc., which are not limited here. For example, it can be understood that the foreground area recognition processing here refers to the process of determining and identifying the foreground area in the target positioning image. The foreground probability refers to the probability that a pixel point is a foreground pixel point (that is, a pixel point belonging to the foreground area). The higher the foreground probability, the greater the probability that the pixel point is a foreground pixel point (that is, the greater the probability that the pixel point belongs to the foreground area). Conversely, the smaller the foreground probability, the smaller the probability that the pixel point is a foreground pixel point (that is, the smaller the probability that the pixel point belongs to the foreground area).

[0135] Optionally, the specific process of the computer device performing foreground area recognition processing on the target positioning image using the target foreground recognition network can be described as follows: the computer device can perform feature extraction processing on the target positioning image based on the target foreground recognition network to obtain image features of the target positioning image; further, the computer device can perform foreground recognition processing based on the image features to obtain a foreground probability for each pixel in the target positioning image. The feature extraction processing can be implemented using a multi-layer neural network. The image features refer to image features used for foreground recognition and can be represented as a feature map.

[0136] Among them, the computer device performs feature extraction processing on the target positioning image based on the target foreground recognition network to obtain the specific process of image features, which can be described as: the computer device can determine multiple initial feature maps corresponding to the target positioning image based on the target foreground recognition network. In the multiple initial feature maps, the feature size of each initial feature map is different. In other words, the embodiment of the present application can perform feature extraction processing of different scales on the target positioning image through the target forward recognition network to obtain multiple initial feature maps of different scales; further, the computer device can perform feature fusion processing on the multiple initial feature maps to obtain the image features of the target positioning image. Among them, different initial feature maps can be obtained by performing convolution processing on the target positioning image based on convolution kernels of different sizes (i.e., the aforementioned different scales). Due to the different sizes of the convolution kernels, the sizes of the initial feature maps obtained by the computer device performing convolution processing on the initial feature maps will also be different. Among them, it can be understood that in the process of performing feature fusion processing on multiple initial feature maps to obtain the image features of the target positioning image, the computer device can specifically perform feature fusion processing on multiple initial feature maps through a neural network layer to obtain the image features of the target positioning image.

[0137] Among them, it can be understood that the specific process of the computer device determining the foreground area corresponding to the target object based on the foreground probability of each pixel point in the target positioning image can be described as: the computer device can determine the pixel points with a foreground probability greater than or equal to the probability threshold in the target positioning image as foreground pixel points, and can determine the foreground area corresponding to the target object based on the foreground pixel points.

[0138] Optionally, when determining the foreground area corresponding to the target object based on the foreground pixels, the computer device may also determine whether each foreground pixel is adjacent to other foreground pixels, and then adjust isolated foreground pixels (i.e., foreground pixels that are discontinuous or non-adjacent to other foreground pixels) to background pixels, so that the foreground area can be determined based on the foreground pixels after removing the isolated foreground pixels. Background pixels refer to pixels belonging to the background area. It should be understood that the background pixels here may refer to pixels other than foreground pixels identified in the target positioning image.

[0139] For example, please refer to Figure 7, which is a schematic diagram of the effect of a target foreground recognition network provided by an embodiment of the present application. When the computer device obtains the target positioning image 701, the target positioning image 701 can be input into the target foreground recognition network 702 shown in Figure 7. In this way, the target foreground recognition network 702 can perform feature extraction processing on the target positioning image through the feature extraction component 721 to obtain the image features of the target positioning image 701. Furthermore, as shown in Figure 7, the computer device can perform foreground recognition processing on the image features of the target positioning image 701 output by the feature extraction component 721 through the foreground recognition component 722 to obtain the foreground area 703 in the target positioning image.

[0140] For ease of understanding, here, taking the target foreground recognition network as a neural network as an example, the specific process of performing foreground recognition processing on the target positioning image based on the target foreground recognition network is explained. Further, please refer to Figure 8, which is a structural diagram of a feature extraction component provided in an embodiment of the present application. The feature extraction component 721 shown in Figure 8 can specifically be the specific structure of the feature extraction component 721 shown in Figure 7 above. In this way, after the computer device inputs the target positioning image 801 into the feature extraction component 721, it can extract multiple initial feature maps corresponding to the target positioning image 801 through the feature extraction component 721 (for example, the multiple initial feature maps 802 shown in Figure 8). It should be understood that the feature sizes of each initial feature map 802 shown in Figure 8 are different; further, as shown in Figure 8, the computer device can perform feature fusion processing on the multiple initial feature maps 802 to obtain the image feature 803 shown in Figure 8. Among them, the initial feature maps of different feature sizes can be obtained after convolution processing is performed on the target positioning image based on convolution kernels of different sizes. Since the sizes of the convolution kernels used for convolution processing are different, the sizes of the initial feature maps obtained by the computer device after convolution processing on the initial feature map through convolution kernels of different sizes will be different.

[0141] Optionally, in another possible implementation, the target foreground recognition network can also be a semantic segmentation network based on HRNetV2 (a neural network for extracting images of different resolutions). Since when feature extraction is performed through HRNet (a neural network), image features with richer features and multiple resolutions can be extracted, the extracted image features of different resolutions can be further connected in parallel through the convolutional layer in the semantic segmentation network to ensure that the image features obtained by feature fusion contain more features that can be used for information exchange and fusion. For example, the image features output by HRNet (a neural network) contain the output of the high-resolution sub-network used for the final prediction, which means that the image features output by HRNet (a neural network) can ensure that the network consistently maintains high-resolution features. Therefore, accurate pixel-level classification can be achieved through HRNet, which greatly improves the stability of the fitted edge curve, and is very suitable for solving the strict specification requirements in industrial quality inspection.

[0142] It is understood that the target foreground recognition network can be a network obtained by training an initial foreground recognition network based on a large amount of sample data. The initial foreground recognition network can refer to a foreground recognition network that has not been trained or has not completed training. The network structure of the initial foreground recognition network is consistent with the network structure of the target foreground recognition network.

[0143] Specifically, the specific process of a computer device training an initial foreground recognition network to obtain a target foreground recognition network can be described as follows: the computer device can obtain a sample object image; the sample object image here is associated with sample foreground area annotation information of the sample object; further, the computer device can obtain the initial foreground recognition network, and input the sample object image into the initial foreground recognition network, and the initial foreground recognition network performs foreground area recognition processing on the sample object image to obtain the foreground probability of each pixel in the sample object image; further, the computer device can iteratively train the initial foreground recognition network based on the foreground probability of each pixel in the sample object image and the sample foreground area annotation information, and determine the initial foreground recognition network after iterative training as the target foreground recognition network.

[0144] The sample object image refers to an image used to train the initial foreground recognition network. The sample object can be an object used to determine the sample object image. In an embodiment of the present application, the area where the sample object is located can be labeled as a sample foreground area in the sample object image. In this case, the sample foreground area labeling information is the label information of the sample foreground area labeled in the sample object image. In addition, it is understood that when the computer device iteratively trains the initial foreground recognition network based on the foreground probability of each pixel in the sample object image and the sample foreground area labeling information, it can first predict and output predicted foreground area label information corresponding to the sample foreground area in the sample object image based on the foreground probability of each pixel in the sample object image. Then, the network loss value of the initial foreground recognition network can be determined based on the predicted predicted foreground area label information and the pre-labeled sample foreground area labeling information. Then, the initial foreground recognition network can be iteratively trained based on the determined network loss value until the network loss value obtained from the iterative training meets the model convergence condition. The network whose network loss value meets the model convergence condition can be determined as the target foreground recognition network.

[0145] It can be understood that the specific implementation method of the initial foreground recognition network performing foreground area recognition processing on the sample object image to obtain the foreground probability of each pixel in the sample object image can refer to the above-mentioned description of the target foreground recognition network performing foreground area recognition processing on the target positioning image to obtain the foreground probability of each pixel in the target positioning image, and will not be repeated here.

[0146] For example, please refer to Figure 9, which is a schematic diagram of the effect of foreground recognition provided by an embodiment of the present application. As shown in Figure 9, the target positioning image 901 includes a plurality of pixels. When the computer device performs foreground area recognition processing on the target positioning image 901 through the target foreground recognition network, it can determine the foreground probability of each pixel being a foreground pixel, and then when the foreground probability is greater than a preset probability threshold, the pixel with a foreground probability greater than a preset probability threshold can be determined as a foreground pixel. For example, in the target positioning image 901 shown in Figure 9, the foreground area of ​​the target object identified from the target positioning image 901 by the target foreground recognition network can be the foreground area 902 shown in Figure 9. At this time, the computer device can refer to the other areas except the foreground area 902 in the target positioning image 901 as background areas. For example, the background area here can be the background area 903 in Figure 9.

[0147] Among them, the specific process of the computer device performing edge curve fitting processing on the pixel points in the foreground area to obtain the edge curve of the target object can be described as: the computer device can determine the edge pixel point set of the target object from the pixel points in the foreground area; the edge pixel point set here may include N edge pixel points; N is a positive integer; further, the computer device can directly perform curve fitting processing based on the N edge pixel points to fit the edge curve of the target object.

[0148] Optionally, in another implementable manner, in order to improve the efficiency of curve fitting to obtain an edge curve, the computer device may also select (for example, randomly select or select based on corresponding pixel selection rules) M edge pixel points from the N edge pixel points included in the edge pixel point set to determine the selected M edge pixel points as M first fitting pixel points; M is a positive integer less than N; further, the computer device may perform curve fitting processing based on the M first fitting pixel points to obtain a fitting curve corresponding to the M first fitting pixel points; further, the computer device may determine the edge curve of the target object based on the fitting curve corresponding to the M first fitting pixel points.

[0149] The edge pixel set refers to a set of edge pixel points, and the edge pixel points refer to pixel points located at the edge of the foreground area. The edge pixel set may include multiple edge pixel points, that is, N edge pixel points.

[0150] Among them, the edge curve refers to the curve obtained by fitting based on the edge pixel points in the foreground area. The edge curve can be a fitted edge straight line, a fitted quadratic curve, a fitted circle, etc., which is not limited here. In other words, the curve type of the edge curve can be a straight line, a quadratic curve, a circle, etc., which is not limited here. It is understandable that when the computer device performs curve fitting processing based on N edge pixel points to obtain the edge curve of the target object, it can specifically use a random sampling consistent linear regression algorithm, a simple linear regression algorithm, and regression using a neural network to perform curve fitting processing, which is not limited here.

[0151] It can be understood that in some scenarios, the edge curve can be an edge straight line, and the target object can be measured based on the edge straight line to obtain a detection value, such as determining the shortest distance from some key positions in the target object to the edge straight line as the detection value, thereby performing object quality management through the measured detection value, such as determining whether the target object meets the specifications based on the detection value.

[0152] Specifically, the edge curve includes an edge straight line; then, the embodiment of the present application may also include the following steps: determining the pixel point to be detected in the target positioning image, and determining the detection value associated with the pixel point to be detected based on the distance from the pixel point to be detected to the edge straight line; performing object quality management on the target object based on the detection value.

[0153] The pixel to be detected may be a pixel corresponding to the position to be detected in the target object. The detection value may be a value obtained by measuring the pixel to be detected based on the edge line, such as the distance between the pixel to be detected and the edge line. It will be understood that the distance between the pixel to be detected and the edge line may refer to the shortest distance between the pixel to be detected and the edge line.

[0154] Among them, the specific process of the computer device performing object quality management on the target object based on the detection value can be described as: the computer device can obtain a reference value range, compare the reference value with the detection value, and obtain a comparison result; if the comparison result indicates that the detection value is within the reference value range, the target object is determined to be a compliant object; conversely, if the comparison result indicates that the detection value is not within the reference value range, the target object is determined to be a non-compliant object. Among them, the compliant object can be an object with compliant dimensions, and the non-compliant object can be an object with non-compliant dimensions. Therefore, the embodiment of the present application can be applied to measure industrial products (such as target objects) in industrial scenarios to detect whether the dimensions of the industrial products are compliant and to achieve quality management of the industrial products.

[0155] It is understandable that in the quality inspection process of the industrial manufacturing industry, dimensional specifications are a very strict standard. In some applications, the algorithm measurement consistency error is required to be less than 2 pixels, and the deviation from the actual value is less than 4 pixels. If the size is incorrect, the assembly of the device may fail, and it may also damage other complete structures of the product. Therefore, in industrial quality inspection, it is necessary to rely on image vision to measure the size of the product. Based on this, in order to measure the size of the product, it is necessary to locate the positions of various key points and lines. Therefore, the image processing scheme involved in the embodiment of the present application can accurately fit the edge curve of the target object, thereby improving the efficiency and accuracy of the quality management of the target object (such as industrial devices and products).

[0156] It is understood that when the edge curve is another type of curve, detection can also be performed based on the edge curve to perform object quality management on the target object based on the detection value. Specifically, the computer device can determine the pixel to be detected in the target positioning image, and determine the detection value associated with the pixel to be detected based on the distance between the pixel to be detected and the edge curve, and then perform object quality management on the target object based on the detection value.

[0157] Among them, the distance from the pixel to be detected to the edge curve can refer to the shortest distance from the pixel to be detected to the edge curve. For example, when the edge curve is a circle, the distance from the pixel to be detected to the edge curve can be determined based on the distance from the point to the center of the circle and the radius of the circle. For example, the absolute value of the difference between the radius of the circle and the distance from the pixel to be detected to the center of the circle is determined as the distance from the pixel to be detected to the edge curve. For another example, when the edge curve is a quadratic curve (such as an ellipse, a hyperbola, etc.), the point closest to the pixel to be detected can be assumed (referred to as the first close distance point), and the tangent h1 of the edge curve is determined through the first close distance point, and the line h2 connecting the first close distance point and the pixel to be detected is determined. It is understood that the tangent line h1 and the connecting line h2 should be perpendicular to each other. The specific position of the first close distance point can be determined by the product of the slope A1 of the tangent line h1 and the slope A2 of the connecting line h2 (for example, by determining the specific position of the first close distance point by multiplying A1 and A2 to -1), and then the distance between the pixel to be detected and the edge curve can be determined based on the distance between the pixel to be detected and the first close distance point. Alternatively, the shortest distance from the pixel to be detected to the edge curve, which is a quadratic curve, can be determined by other methods. For example, assuming that a circle is determined with the pixel to be detected as the center, when the circle is tangent to the edge curve, the radius of the circle is the shortest distance from the pixel to be detected to the edge curve. For example, when the edge curve is of other curve types, such as when the edge curve is a curve determined based on a continuously differentiable function, the edge curve can be differentiated to determine the tangent of a certain point on the edge curve (recorded as the first curve point). This is because the tangent at a certain point on the curve is the straight line represented by the derivative of the curve at that point, and the line connecting the first curve point and the pixel point to be detected is determined. When the line and the tangent of the first curve point are perpendicular to each other, the shortest distance from the pixel point to be detected to the edge curve is obtained.

[0158] It is understandable that there may be other measurement methods according to actual needs. For example, when the edge curve is an edge circle (i.e., the curve type is an edge curve of a circle), the radius of the edge circle is determined as the detection value, and the target object is subject to object quality management based on the detection value. In some scenarios, the shape of the portion to be detected in the target positioning area is a circle, and its radius needs to be measured. Then, through the embodiments of the present application, a circular edge curve can be fitted, and the radius of the object can be determined based on the fitted edge curve. For another example, when the edge curve is an ellipse, the distance between the pixel to be detected and the straight line on which the major axis of the ellipse is located can be determined as the detection value, and the target object is subject to object quality management based on the detection value. For another example, when the edge curve is an ellipse, the length of the major axis or minor axis of the ellipse can be determined as the detection value, and the target object is subject to object quality management based on the detection value. In some scenarios, the shape of the object in the target positioning area is an ellipse, and the length of its major axis and minor axis needs to be measured. Then, through the embodiments of the present application, an elliptical edge curve can be fitted, and the length of the major axis and minor axis of the object can be determined based on the fitted edge curve.

[0159] In an embodiment of the present application, when a business image containing a target object is acquired, a template image for assisting in locating the target object is acquired, and an object detection area can be determined from the template image. It should be understood that the area size of the object detection area here is smaller than the area size of the business image. In this way, when an area matching the object detection area is found in the business image, the embodiment of the present application can quickly determine the found area as the initial positioning area of ​​the target object, thereby determining a pixel position transformation relationship based on the pixel positions of the pixels in the object detection area and the pixel positions of the pixels in the initial positioning area. Furthermore, when a local reference area is determined from the template image (the local reference area here refers to a local area pre-marked in the template image), the embodiment of the present application can directly determine the local mapping area corresponding to the local reference area in the business image based on the pixel position transformation relationship (for example, the inverse matrix of the above-mentioned pixel position transformation matrix), and then the local mapping area can be used as the target positioning area of ​​the target object. Furthermore, in an embodiment of the present application, a foreground area corresponding to the target object can be identified in the target positioning image corresponding to the target positioning area, and then edge curve fitting processing can be performed on the pixels in the foreground area to obtain the edge curve of the target object. It can be seen that the embodiment of the present application can first quickly determine an initial positioning area in the business image based on the object detection area in a template image, and then construct a pixel position transformation matrix for transforming (for example, forward transforming) the pixel points in the initial positioning area to the pixel position in the object detection area through the pixel positions of the pixel points in the initial positioning area and the pixel positions of the pixel points in the object detection area, so that the pixel position transformation relationship represented by the pixel position transformation matrix can be determined by the forward transformation relationship. In this way, when there is a pre-marked local area (that is, the aforementioned local reference area) in the template image, the pixel positions of the key pixels in the local reference area can be transformed (for example, inversely transformed) through the inverse transformation relationship in the pixel position transformation relationship (for example, the inverse matrix of the pixel position transformation matrix) to obtain the local mapping area corresponding to the local reference area, and then the local mapping area can be quickly used as the target positioning area located in the business image, so that when the foreground area is subsequently identified in the target positioning image corresponding to the target positioning area, the edge curve can be quickly and accurately determined based on the foreground area.

[0160] Further, referring to FIG10 , FIG10 is a flow chart of an image processing method provided in an embodiment of the present application. The method can be executed by a computer device. The method can include at least the following steps S201 to S208.

[0161] Step S201: Acquire a business image containing a target object, acquire a template image for assisting in positioning the target object, and determine an object detection area corresponding to the template object from the template image;

[0162] The template object refers to an object existing in the template image and having the same object type as the target object, and the area size of the object detection area is smaller than the area size of the service image.

[0163] Step S202: Search for an area in the business image that matches the object detection area, determine the found area as the initial positioning area of ​​the target object, and determine the pixel position transformation relationship based on the pixel positions of the pixel points in the object detection area and the pixel positions of the pixel points in the initial positioning area.

[0164] Step S203: When the local reference area is determined from the template image, a local mapping area corresponding to the local reference area is determined in the service image based on the pixel position transformation relationship, and the local mapping area is used as the target positioning area of ​​the target object.

[0165] The local mapping area refers to the area obtained by performing pixel position transformation on the pixel points in the local reference area based on the pixel position transformation relationship;

[0166] Step S204: Identify a foreground area corresponding to the target object in the target positioning image corresponding to the target positioning area.

[0167] The implementation method of steps S201-S204 may refer to the relevant description of the above steps S101-S104, which will not be repeated here.

[0168] Step S205: determining an edge pixel point set of the target object from the pixels in the foreground area; the edge pixel point set includes N edge pixel points;

[0169] Wherein, N is a positive integer, the edge pixel set refers to a set consisting of edge pixels, and the edge pixel points refer to pixels located at the edge of the foreground area. The edge pixel set may include multiple edge pixel points, that is, the multiple edge pixel points here may be N edge pixel points.

[0170] It can be understood that the specific process of the computer device determining the edge pixel point set of the target object from the pixel points in the foreground area can be described as: the computer device can determine the starting image boundary from the target positioning image, where the starting image boundary includes R pixel points, and the R pixel points include the sth pixel point, and s is a positive integer less than or equal to R; further, the computer device can take the sth pixel point as the starting point, traverse each pixel point in turn in the direction perpendicular to the starting image boundary until it traverses to the first foreground pixel point, and determine the determined first foreground pixel point as the edge pixel point corresponding to the sth pixel point; further, the computer device can determine multiple edge pixel points corresponding to the starting image boundary based on the edge pixel point corresponding to the sth pixel point, and determine the edge pixel point set based on the multiple edge pixel points corresponding to the starting image boundary.

[0171] It is understandable that the starting image boundary refers to the image boundary used to determine the edge pixel points in the corresponding orientation in the target positioning image. The image boundary of an image can be divided into an upper boundary (also called a first boundary), a lower boundary (also called a second boundary), a left boundary (also called a second boundary) and a right boundary (also called a second boundary). The starting image boundary can be the image boundary in the orientation where the edge pixel points need to be determined. For example, if the edge pixel points at the lower edge of the target object need to be determined, the lower boundary in the target positioning image can be determined as the starting image boundary. It is understandable that if it is necessary to determine the edge pixel points in multiple orientations (such as the four orientations of up, down, left and right), the multiple image boundaries in the target positioning image are all determined as the starting image boundary, thereby determining the multiple edge pixel points corresponding to each starting image boundary, and then the multiple edge pixel points corresponding to each starting image boundary can be deduplicated to obtain an edge pixel point set. It is understandable that the determination of the starting image boundary can be determined based on actual needs and is not limited here.

[0172] For example, please refer to Figure 11, which is a process for determining an edge pixel point set provided by an embodiment of the present application. As shown in Figure 11, the foreground area of ​​the target object identified by the computer device in the target positioning image can be a foreground area 1102. Further, the computer device can collectively refer to the pixels in the foreground area 1102 as the edge pixel point set of the target object. Specifically, the starting image boundary determined by the computer device in the target positioning image can be the starting image boundary 1101 shown in Figure 11. As shown in Figure 11, the starting image boundary 1101 includes multiple pixel points. Here, taking the s-th pixel point shown in Figure 11 as an example, the specific process of determining the edge pixel point corresponding to the s-th pixel point is explained. Here, the sth pixel point can be used as the starting point, and each pixel point can be traversed in sequence to the left along the direction of the arrow shown in Figure 11 until the first foreground pixel point is determined. The first foreground pixel point here can be the edge pixel point corresponding to the sth pixel point shown in Figure 11. Similarly, the processing process of the sth pixel point can be referred to, and the edge pixel point corresponding to each pixel point on the starting image boundary 1101 can be traversed to determine, thereby constructing an edge pixel point set. For example, the set of edge pixel points determined by the starting image boundary 1101 can be the edge pixel point set 1103 shown in Figure 11. It should be understood that the edge pixel point set 1103 can be the edge pixel point set determined from the foreground area 1102.

[0173] It can be understood that, further, the computer device can also determine other image boundaries as the starting image boundaries, and then determine the edge pixel points corresponding to the pixel points on the other image boundaries, and can construct other edge pixel point sets through the edge pixel points corresponding to the pixel points on the other image boundaries. Then, the computer device can deduplicate the edge pixel points in these determined edge pixel point sets to obtain the complete edge pixel points (for example, N edge pixel points) of the target object in the target positioning image, so the pixel point set composed of these complete edge pixel points can be collectively referred to as the edge pixel point set of the target object.

[0174] Step S206 : Select M edge pixel points from the N edge pixel points included in the edge pixel point set, and determine the selected M edge pixel points as M first fitting pixel points; M is a positive integer less than N.

[0175] It is understood that, in the embodiment of the present application, the computer device may randomly select M edge pixels from the N edge pixels included in the edge pixel set, and determine the M randomly selected edge pixels as M first fitting pixels. It should be understood that the first fitting pixel can be a pixel selected from the edge pixel set specifically for curve fitting. For example, the first fitting pixel can be an edge pixel randomly selected from the N edge pixels included in the edge pixel set.

[0176] It should be understood that in order to obtain an edge curve through fitting, the number M should be at least 2, in other words, M is a positive integer greater than or equal to 2. In actual application scenarios, the specific value of M can be determined according to actual needs.

[0177] Optionally, in another possible implementation, the number of randomly selected pixels can be preset, or the ratio of the number of randomly selected pixels to the edge pixel set can be preset, and then the computer device can determine the number of randomly selected pixels required based on the preset ratio.

[0178] Step S207 : performing curve fitting processing based on the M first fitting pixel points to obtain a fitting curve corresponding to the M first fitting pixel points.

[0179] The fitting curve may be a curve obtained by performing a curve fitting process based on fitting pixels (e.g., first fitting pixels) randomly selected from the edge pixel set. The fitting pixels (e.g., first fitting pixels) refer to edge pixels used to fit the fitting curve. It is understood that performing a curve fitting process may also be referred to as simply performing fitting.

[0180] It can be understood that the specific process of the computer device performing curve fitting processing based on M first fitting pixel points to obtain the fitting curve corresponding to the M first fitting pixel points can be described as: the computer device can obtain the curve function to be fitted, and can perform curve fitting processing based on the M first fitting pixel points and the curve function to be fitted to obtain the fitting curve corresponding to the M first fitting pixel points.

[0181] Among them, the curve function to be fitted can be a curve function to be obtained. For example, if the curve function to be fitted is a straight line function, the fitting curve determined by the curve function to be fitted can be a straight line; alternatively, if the curve function to be fitted is a quadratic curve function, the fitting curve determined by the curve function to be fitted can be a quadratic curve; alternatively, if the curve function to be fitted is a circle function, the fitting curve determined by the curve function to be fitted can be a circle. In other words, the embodiment of the present application can determine the corresponding curve function to be fitted according to the type of curve that actually needs to be fitted (such as a straight line, a quadratic curve, a circle, etc.), so that the corresponding type of fitting curve can be determined, and then the corresponding type of edge curve can be determined. This means that in the embodiment of the present application, the type of curve that needs to be fitted can be determined according to actual needs.

[0182] Step S208: Determine the edge curve of the target object based on the fitting curves corresponding to the M first fitting pixel points.

[0183] It can be understood that, the edge curve of the target object is determined based on the fitting curve corresponding to the M first fitting pixel points. The fitting curve corresponding to the M first fitting pixel points can be directly determined as the edge curve of the target object, or the above steps S206-S207 can be repeatedly performed to determine the fitting curve corresponding to another group of M fitting pixel points (such as M second fitting pixel points), and the edge curve of the target object is determined from multiple fitting curves. This is not limited here.

[0184] It is understood that the specific process of the computer device determining the edge curve of the target object from multiple fitting curves can be described as follows: the computer device can determine the fitting curve with the largest number of in-group pixels among the multiple fitting curves as the edge curve, where the number of in-group pixels refers to the number of in-group pixels associated with the fitting curve. In-group pixels refer to edge pixels whose distance from the fitting curve is less than or equal to a distance threshold.

[0185] It can be understood that when a computer device determines the edge curve of a target object from multiple fitting curves, it can also be determined when the number of times the fitting curve is repeatedly calculated (i.e., the number of curve fitting times) reaches a fitting times threshold, or it can also be determined when the number of pixels in the inner group of the calculated fitting curve is greater than or equal to the number threshold.

[0186] Specifically, the specific process of the computer device determining the edge curve of the target object based on the fitting curve corresponding to the M first fitting pixel points can be described as: the computer device can determine the fitting curve corresponding to the M first fitting pixel points as the first fitting curve, and calculate the distance of each edge pixel point among the N edge pixel points to the first fitting curve; further, the computer device can determine the inner group pixel points of the first fitting curve from the N edge pixel points based on the distance to the first fitting curve; the inner group pixel points here refer to edge pixel points whose distance from the fitting curve is less than or equal to a distance threshold; further, the computer device can count the number of pixels of the inner group pixel points associated with the first fitting curve, and determine the number of pixels obtained by counting as the number of inner group pixels of the first fitting curve; further, if the number of inner group pixels of the first fitting curve is greater than or equal to the number threshold, the computer device can determine the first fitting curve as the edge curve of the target object.

[0187] Among them, the first fitting curve is a fitting curve obtained by fitting based on M first fitting pixel points. The inner group pixel point refers to an edge pixel point whose distance from the fitting curve is less than or equal to the distance threshold. It can be understood that the distance threshold is the minimum value that the distance from the fitting curve needs to meet when the edge pixel point is determined to be an inner group pixel point. In other words, when the distance between the edge pixel point and the fitting curve is less than or equal to the distance threshold, the edge pixel point is determined to be an inner group pixel point. It can be understood that the edge pixel points other than the inner group pixel points in the edge pixel point set can be called outlier pixels (also called outliers).

[0188] It is understandable that the distance from the edge pixel point to the first fitting curve refers to the shortest distance from the edge pixel point to the first fitting curve. For example, when the first fitting curve is a straight line, the distance from the edge pixel point to the first fitting curve can be determined by calculating the shortest distance from the point to the straight line. For another example, when the first fitting curve is a circle, the distance from the edge pixel point to the first fitting curve can be determined based on the distance from the point to the center of the circle and the radius of the circle. For example, the absolute value of the difference between the radius of the circle and the distance from the edge pixel point to the center of the circle is determined as the distance from the edge pixel point to the first fitting curve. For another example, when the first fitting curve is a quadratic curve (such as an ellipse, a hyperbola, etc.), the point closest to the edge pixel point can be assumed (referred to as the second closest point), and the tangent line h1 of the first fitting curve is determined through the second closest point, and the line h2 connecting the second closest point and the edge pixel point is determined. It is understandable that the tangent line h1 and the connecting line h2 should be perpendicular to each other. The specific position of the second closest point can be determined by the product of the slope A1 of the tangent line h1 and the slope A2 of the connecting line h2 (for example, by determining the specific position of the second closest point by multiplying A1 and A2 to -1), and then the distance between the edge pixel point and the first fitting curve can be determined based on the distance between the edge pixel point and the second closest point. Alternatively, the shortest distance from the edge pixel point to the first fitting curve, which is a quadratic curve, can be determined by other methods. For example, assuming that a circle is determined with the edge pixel point as the center, when the circle is tangent to the first fitting curve, the radius of the circle is the shortest distance from the edge pixel point to the first fitting curve. For another example, when the first fitting curve is of other curve types, such as when the first fitting curve is a curve determined based on a continuously differentiable function, the first fitting curve can be differentiated to determine the tangent of a certain point on the first fitting curve (recorded as the second curve point). This is because the tangent at a certain point on the curve is the straight line represented by the derivative of the curve at that point, and the line connecting the second curve point and the edge pixel point is determined. When the line and the tangent of the second curve point are perpendicular to each other, the shortest distance from the edge pixel point to the first fitting curve is obtained.

[0189] It is understood that the number of in-group pixels refers to the number of in-group pixels of the fitted curve. It is understood that the greater the number of in-group pixels of a fitted curve, the more likely that the fitted curve is to best fit the distribution of edge pixels. The number threshold can be the minimum value that the number of in-group pixels must meet to determine the first fitted curve as an edge curve. In other words, when the number of in-group pixels of the first fitted curve is greater than or equal to the number threshold, the first fitted curve is determined to be the edge curve of the target object. In other words, when the number of in-group pixels of the first fitted curve meets the requirement, the first fitted curve can be directly determined as the edge curve. Furthermore, if the first fitted curve does not meet the requirement (i.e., the number of in-group pixels corresponding to the first fitted curve is less than the number threshold), the step of selecting (e.g., randomly selecting) M edge pixels from the N edge pixel set can be repeated, so that another fitted curve can be fitted based on the newly selected (e.g., randomly selected) M edge pixels, and the edge curve of the target object can be determined from the fitted curves each time.

[0190] Specifically, the specific process of the computer device determining the edge curve of the target object based on the fitting curve corresponding to the M first fitting pixel points can also be described as: when determining the fitting curve corresponding to the M first fitting pixel points, the computer device can record the number of curve fittings associated with the edge curve, and determine the number of inner group pixels of the fitting curve corresponding to the M first fitting pixel points as the initial number of inner group pixels; wherein the number of inner group pixels is the number of pixels of the inner group pixel points of the fitting curve; further, when the number of curve fittings does not reach the fitting number threshold, the computer device can notify the execution of the step of selecting (for example, randomly selecting) M edge pixel points from the N edge pixel points included in the edge pixel point set; and determine the selected M edge pixel points as M The second fitting pixel points can then be used to perform curve fitting processing based on the M second fitting pixel points to obtain fitting curves corresponding to the M second fitting pixel points; further, the computer device can obtain the number of inner group pixels of the fitting curve corresponding to the M second fitting pixel points, and determine the number of inner group pixels of the fitting curve corresponding to the M second fitting pixel points as the number of inner group pixels to be compared; further, the computer device can determine the target number of inner group pixels based on the number of inner group pixels to be compared and the initial number of inner group pixels; here, the target number of inner group pixels is the largest number of inner group pixels between the number of inner group pixels to be compared and the initial number of inner group pixels; further, the computer device can determine the edge curve of the target object based on the fitting curve corresponding to the target number of inner group pixels.

[0191] It is understood that the curve fitting count refers to the number of times a curve fitting process is performed on M randomly selected edge pixels from a set of edge pixels to obtain a fitted curve. For example, when obtaining a fitted curve corresponding to the M first fitting pixels, the curve fitting count may be 1. Each subsequent calculation of the fitted curve may update the curve fitting count, for example, by incrementing the curve fitting count by 1 until the updated curve fitting count reaches a fitting count threshold. Within the preset fitting count threshold, the edge curve of the target object may then be determined from the multiple fitted curves obtained by fitting.

[0192] The initial number of inner-group pixels refers to the number of inner-group pixels of the fitting curve corresponding to the first fitting pixel point.

[0193] It is understood that the fitting number threshold refers to the maximum number of times the curve fitting is repeated to obtain the fitted curve. In other words, if the number of curve fitting times does not reach the fitting number threshold, the step of randomly selecting M edge pixels and performing curve fitting based on the selected M edge pixels to obtain the fitted curve needs to be repeated until the number of curve fitting times reaches the fitting number threshold (or the number of pixels in the inner group of the fitted curve is greater than or equal to the number threshold).

[0194] The second fitting pixel points may be M edge pixel points randomly selected from the edge pixel set after the first fitting curve is determined. It is understood that the specific implementation of determining the fitting curve corresponding to the M second fitting pixel points can refer to the description of determining the fitting curve corresponding to the M first fitting pixel points above, and is not further described here.

[0195] It is understood that the number of inner group pixels to be compared refers to the number of inner group pixels of the fitting curve corresponding to the M second fitting pixels. The target number of inner group pixels is the largest number of inner group pixels between the number of inner group pixels to be compared and the initial number of inner group pixels.

[0196] It can be understood that when a computer device determines the edge curve of a target object based on a fitting curve corresponding to the number of pixels in the target inner group, the fitting curve corresponding to the number of pixels in the target inner group can be determined as the edge curve of the target object. In other words, the fitting curve with the largest number of pixels in the inner group among the fitting curves obtained each time is determined as the edge curve.

[0197] It can be understood that the specific process of the computer device determining the edge curve of the target object based on the fitting curve corresponding to the number of pixels in the target group can also be described as: the computer device can update the number of curve fittings when determining the fitting curves corresponding to the M second fitting pixel points; further, when the number of curve fittings after the update does not reach the fitting number threshold, the computer device notifies the execution of the step of selecting (for example, randomly selecting) M edge pixel points from the N edge pixel points included in the edge pixel point set; further, the computer device can determine the selected M edge pixel points as M third fitting pixel points, and based on the M The computer device may perform curve fitting processing on the third fitting pixel points to obtain fitting curves corresponding to the M third fitting pixel points; further, the computer device may obtain the number of inner group pixels of the fitting curve corresponding to the M third fitting pixel points, and update the number of inner group pixels to be compared based on the number of inner group pixels of the fitting curve corresponding to the M third fitting pixel points; further, the computer device may update the number of target inner group pixels based on the updated number of inner group pixels to be compared; it should be understood that the updated number of target inner group pixels is the largest number of inner group pixels between the updated number of inner group pixels to be compared and the target number of inner group pixels before the update.

[0198] The third fitting pixel points may be the M edge pixel points randomly selected from the edge pixel set after the second fitting curve is determined. It is understood that the specific implementation of determining the fitting curves corresponding to the M third fitting pixel points can refer to the description of determining the fitting curves corresponding to the M first fitting pixel points, and is not further described here.

[0199] It is understood that the updating of the number of inner-group pixels to be compared based on the number of inner-group pixels of the fitting curve corresponding to the M third fitting pixel points may specifically be performed by determining the number of inner-group pixels of the fitting curve corresponding to the M third fitting pixel points as the updated number of inner-group pixels to be compared. Furthermore, the updating of the target number of inner-group pixels based on the updated number of inner-group pixels to be compared may specifically be performed by determining the largest number of inner-group pixels between the updated number of inner-group pixels to be compared and the target number of inner-group pixels before the update as the updated number of inner-group pixels.

[0200] It is understandable that in the process of repeatedly selecting M edge pixels from the edge pixel set and fitting the fitting curve based on the selected M edge pixels, the number of pixels in the inner group to be compared and the number of pixels in the target inner group can be continuously updated. For example, assuming the fitting number threshold is 5 times, M edge pixels are selected from the edge pixel set for the first time, and the selected M edge pixels are determined as the first fitting pixels. Fitting curve 1 is obtained based on the first fitting pixels, and the number of pixels in the inner group corresponding to fitting curve 1 is determined as the initial number of pixels in the inner group. In this case, the number of curve fitting times is 1. When the number of inner group images of the fitting curve 1 is less than the number threshold and the number of curve fittings is less than the fitting number threshold (i.e., 5 times), M edge pixel points can be reselected from the edge pixel point set for the second time, and the reselected M edge pixel points are determined as the second fitting pixel points. The fitting curve 2 is obtained based on the second fitting pixel points, and the number of inner group pixels corresponding to the fitting curve 2 is determined as the number of inner group pixels to be compared. The number of curve fittings at this time is 2, and then the largest number of inner group pixels between the initial number of pixels at this time and the number of inner group pixels to be compared can be determined as the target number of inner group pixels. Furthermore, when the number of inner group images of the fitting curve 2 is less than the number threshold, and the number of curve fittings at this time is less than the fitting number threshold (i.e., 5 times), M edge pixel points can be reselected from the edge pixel point set for the third time, and the M edge pixel points reselected again can be determined as the third fitting pixel points. The fitting curve 3 is obtained based on the third fitting pixel points, and then the number of inner group pixels to be compared can be updated by the number of inner group pixels corresponding to the fitting curve 3 (in other words, the number of inner group pixels corresponding to the fitting curve 3 is determined as the new number of inner group pixels to be compared), and the target number of inner group pixels is updated by the new number of inner group pixels to be compared (in other words, the largest number of inner group pixels between the number of inner group pixels to be compared and the target number of inner group pixels is determined as the new target number of inner group pixels). At this time, the number of curve fittings is 3. Similarly, the steps of randomly selecting M edge pixels and determining the fitting curve based on the selected M edge pixels are repeated, and the target in-group pixel number is updated until the fitting end condition is reached. At this time, the updated target in-group pixel number is the largest in-group pixel number among the in-group pixel numbers of the fitting curve obtained by each fitting.

[0201] It is understandable that the condition for not repeatedly executing the step of selecting M pixels from the edge pixel set to determine the fitting curve (i.e., the fitting end condition) may be that the number of curve fitting times reaches the fitting times threshold. In other words, the condition for repeatedly executing the step of selecting M pixels from the edge pixel set to determine the fitting curve may be that the number of curve fitting times does not reach the fitting times threshold.

[0202] Specifically, the specific process of the computer device determining the edge curve of the target object based on the fitting curve corresponding to the number of pixels in the target group can also be described as: the computer device can update the number of curve fittings when determining the fitting curves corresponding to M second fitting pixel points; further, the computer device can determine the fitting curve corresponding to the number of pixels in the target group as the edge curve of the target object when the updated number of curve fittings reaches a fitting number threshold.

[0203] Here, updating the number of curve fitting times may be performed by adding 1 to the number of curve fitting times. It is understandable that the target number of in-group pixels may be used to record the largest number of in-group pixels among the numbers of in-group pixels of the multiple fitting curves previously calculated, and then determining the fitting curve corresponding to the target number of in-group pixels as the edge curve of the target object may be performed by determining the fitting curve with the largest number of in-group pixels among the fitting curves calculated each time as the edge curve.

[0204] Optionally, the fitting end condition may also be that the number of pixels in the inner group of the fitted curve is greater than or equal to the number threshold, or the number of curve fits reaches the fitting number threshold. In other words, when the number of pixels in the inner group of the fitted curve is greater than or equal to the number threshold, or the number of curve fits reaches the fitting number threshold, the step of selecting M pixels from the edge pixel set to determine the fitted curve is no longer repeated, and the fitted curve corresponding to the current largest number of pixels in the inner group is determined as the edge curve. It is understandable that the condition for repeatedly selecting M pixels from the edge pixel set to determine the fitted curve may also be that the number of curve fits does not reach the fitting number threshold and the number of pixels in the inner group of the fitted curve is less than the number threshold, which is not limited here. In other words, when the number of curve fits does not reach the fitting number threshold and the number of pixels in the inner group of the fitted curve is less than the number threshold, the step of selecting M pixels from the edge pixel set to determine the fitted curve may be repeated; or, when the number of curve fits does not reach the fitting number threshold, the step of selecting M pixels from the edge pixel set to determine the fitted curve may be repeated.

[0205] Optionally, the embodiment of the present application may also record the number of inner-group pixels of the fitted curve each time M edge pixel points are determined from the edge pixel point set and a fitting curve is obtained based on the determined M edge pixel points, and then traverse the number of inner-group pixels of each fitting curve, determine the maximum number of inner-group pixels, and determine the fitting curve corresponding to the maximum number of inner-group pixels as the edge curve.

[0206] For example, the process of determining the edge curve is explained here with reference to the diagram. Please refer to Figure 12, which is a flow chart of determining an edge curve provided by an embodiment of the present application. As shown in Figure 12, when the computer device needs to determine the edge curve, it can determine N edge pixel points in the target positioning image based on the foreground area in the target positioning image (i.e., step S1201); then, the computer device can randomly select M fitting pixel points from the N edge pixel points (i.e., step S1202), and then determine the fitting curve based on the M fitting pixel points, count the number of inner group pixel points of the fitting curve, and record the number of curve fitting times (i.e., step S1203). Furthermore, the computer device can determine whether the number of inner group pixel points is greater than or equal to the number threshold (i.e., step S1204). If the number of inner group pixel points is greater than or equal to the number threshold, the fitting curve corresponding to the maximum number of inner group pixels can be determined as the edge curve (i.e., step S1206). In other words, the fitting curve with the number of inner group pixel points greater than or equal to the number threshold at this time is determined as the edge curve.

[0207] Optionally, as shown in FIG12 , if the number of inner group pixels is less than or equal to the number threshold, it may be determined whether the number of curve fitting times reaches the fitting times threshold (i.e., step S1205). If the number of curve fitting times reaches the fitting times threshold, the fitting curve corresponding to the maximum number of inner group pixels may be determined as the edge curve (i.e., step S1206). The maximum number of inner group pixels may be the target inner group pixel points after updating the target inner group pixel points based on the inner group pixel points of the fitting curve when the number of curve fitting times reaches the fitting times threshold, or in other words, the largest inner group pixel point among the inner group pixel points of the fitting curve obtained by each fitting.

[0208] Optionally, as shown in FIG12 , if the curve fitting times do not reach the fitting times threshold, steps S1202 - S1206 may be repeated until the curve fitting times reaches the fitting times threshold, or the number of in-group pixels is greater than or equal to the number threshold.

[0209] Thus, it can be seen that the embodiment of the present application can first quickly determine an initial positioning area in the business image based on the object detection area in a template image, and then determine the pixel position transformation relationship based on the initial positioning area and the object detection area, and perform pixel position transformation on the local area pre-marked in the template image (that is, the aforementioned local reference area) to obtain the local mapping area corresponding to the local reference area, and then quickly use the local mapping area as the target positioning area located in the business image, so that when the foreground area is subsequently identified in the target positioning image corresponding to the target positioning area, the edge pixel point set of the target object can be determined from the pixel points of the foreground area, and then the edge pixel point set can be randomly selected from the N pixel points contained in the edge pixel point set. The machine selects M edge pixel points, and uses the M randomly selected edge pixel points as the M first fitting pixel points to be involved in the curve fitting process. In this way, the edge curve can be quickly and accurately determined through the fitting curve corresponding to the M first fitting pixel points. In other words, in the process of fitting the edge curve in the embodiment of the present application, multiple different fitting curves can be fitted by randomly selecting M fitting pixel points from the edge pixel point set each time, and then the edge curve of the target object can be determined from the multiple different fitting curves obtained by fitting. In this way, not only can the situation where multiple edge curves that meet the fitting conditions but have large actual differences be reduced, but also the accuracy and stability of the edge curve of the target object obtained by fitting can be improved.

[0210] Please refer to Figure 13, which is a schematic diagram of the structure of an image processing device provided in an embodiment of the present application. As shown in Figure 13, the image processing device 1 can be a computer program (including program code) running on a computer device, for example, the image processing device 1 is an application software; it can be understood that the image processing device 1 can be used to execute the corresponding steps of the image processing method provided in an embodiment of the present application. As shown in Figure 13, the image processing device 1 can include: an image acquisition module 11, a transformation relationship determination module 12;

[0211] An image acquisition module 11 is configured to acquire a business image containing a target object, acquire a template image for assisting in locating the target object, and determine an object detection area corresponding to the template object from the template image; the template object is an object of the same object type as the target object existing in the template image, and the area size of the object detection area is smaller than the area size of the business image;

[0212] a transformation relationship determination module 12 for searching an area in the service image that matches the object detection area, determining the found area as an initial positioning area of ​​the target object, and determining a pixel position transformation relationship based on the pixel positions of the pixels in the object detection area and the pixel positions of the pixels in the initial positioning area;

[0213] The region mapping module 13 is configured to determine, when determining the local reference region from the template image, a local mapping region corresponding to the local reference region in the service image based on the pixel position transformation relationship, and use the local mapping region as the target positioning region of the target object; the local mapping region is the region obtained by performing pixel position transformation on the pixels in the local reference region based on the pixel position transformation relationship;

[0214] The curve fitting module 14 is used to identify the foreground area corresponding to the target object in the target positioning image corresponding to the target positioning area, and perform edge curve fitting processing on the pixels in the foreground area to obtain the edge curve of the target object.

[0215] The transformation relationship determination module 12 includes a sliding window unit 121 , a similarity calculation unit 122 , and a region determination unit 123 .

[0216] The sliding window unit 121 is configured to determine a sliding window for window sliding in the service image based on the size of the object detection area, and determine the position of the sliding window in the service image as the to-be-matched area in the service image; the size of the to-be-matched area is consistent with the size of the object detection area;

[0217] A similarity calculation unit 122 is configured to determine the regional similarity between the to-be-matched region and the object detection region by comparing the pixel value of each pixel point in the to-be-matched region with the pixel value of each pixel point in the object detection region;

[0218] The region determining unit 123 is configured to determine, in the service image, a region that matches the object detection region based on region similarity.

[0219] The processing procedures of the sliding window unit 121 , the similarity calculation unit 122 , and the region determination unit 123 may refer to the relevant description in the embodiment of FIG3 , and are not described in detail here.

[0220] The area to be matched includes a first sliding window area and a second sliding window area; the first sliding window area is an area determined when the sliding window slides to the first window position; the second sliding window area is an area determined when the sliding window slides to the second window position; the second window position is the next window position after the first window position;

[0221] The similarity calculation unit 122 includes: a first similarity calculation subunit 1221, a second similarity calculation subunit 1222, and a similarity determination subunit 1223;

[0222] A first similarity calculation subunit 1221 is configured to determine a regional similarity between the first sliding window area and the object detection area based on a pixel value of each pixel point in the first sliding window area and a pixel value of each pixel point in the object detection area, and determine the regional similarity between the first sliding window area and the object detection area as a first regional similarity;

[0223] a second similarity calculation subunit 1222, configured to determine a regional similarity between the second sliding window area and the object detection area based on a pixel value of each pixel point in the second sliding window area and a pixel value of each pixel point in the object detection area, and determine the regional similarity between the second sliding window area and the object detection area as a second regional similarity;

[0224] The similarity determination subunit 1223 is configured to determine the region similarity between the to-be-matched region and the object detection region based on the first region similarity and the second region similarity.

[0225] The processing of the first similarity calculation subunit 1221 , the second similarity calculation subunit 1222 , and the similarity determination subunit 1223 may refer to the relevant description in the embodiment of FIG. 3 , and will not be described in detail here.

[0226] The region determination unit 123 is specifically configured to: a target similarity determination subunit 1231 and a region determination subunit 1232;

[0227] A target similarity determination subunit 1231 is configured to determine a target region similarity based on the first region similarity and the second region similarity, wherein the target region similarity is a region similarity having a maximum value determined between the first region similarity and the second region similarity;

[0228] The region determination subunit 1232 is configured to determine, in the service image, a region that matches the object detection region based on the to-be-matched region corresponding to the target region similarity.

[0229] The processing procedures of the target similarity determination subunit 1231 and the region determination subunit 1232 may refer to the relevant description in the embodiment of FIG3 , and are not described in detail here.

[0230] The first sliding window area includes the i-th pixel point, and the object detection area includes the j-th pixel point; i and j are consistent positive integers, and the pixel position of the i-th pixel point in the first sliding window area is consistent with the pixel position of the j-th pixel point in the object detection area;

[0231] The first similarity calculation subunit 1221 is specifically configured to:

[0232] Calculating the average value of the pixel values ​​of the pixels in the first sliding window area and determining the average value as the first pixel average value of the first sliding window area;

[0233] performing mean calculation on the pixel values ​​of the pixels in the object detection area and determining the average value obtained as a second pixel mean value of the object detection area;

[0234] Determine the difference between the pixel value of the i-th pixel in the first sliding window area and the first pixel mean as the first difference corresponding to the i-th pixel, and determine the difference between the pixel value of the j-th pixel in the object detection area and the second pixel mean as the second difference corresponding to the j-th pixel;

[0235] Based on the first difference value corresponding to the i-th pixel point and the second difference value corresponding to the j-th pixel point, the regional similarity between the first sliding window area and the object detection area is determined.

[0236] The transformation relationship determination module 12 includes: a key pixel point determination unit 124, a related pixel point determination unit 125, a pixel point pair determination unit 126, a pixel position determination unit 127, and a transformation matrix determination unit 128;

[0237] The key pixel point determination unit 124 is configured to determine K key pixel points from the initial positioning area; K is a positive integer greater than 1;

[0238] The associated pixel point determination unit 125 is used to determine K associated pixel points corresponding to K key pixel points in the object detection area; one key pixel point corresponds to one associated pixel point;

[0239] The pixel pair determining unit 126 is configured to construct K pixel pairs based on the K key pixels and the K associated pixels; a pixel pair includes a key pixel and a corresponding associated pixel;

[0240] A pixel position determination unit 127 is configured to determine the pixel positions of K key pixels and the pixel positions of K associated pixels in a target coordinate system;

[0241] The transformation matrix determining unit 128 is configured to calculate a pixel position transformation matrix for representing a pixel position transformation relationship based on the pixel positions of the key pixels and the pixel positions of the associated pixels included in each of the K pixel pairs.

[0242] Among them, the processing procedures of the key pixel point determination unit 124, the associated pixel point determination unit 125, the pixel point pair determination unit 126, the pixel position determination unit 127, and the transformation matrix determination unit 128 can refer to the relevant description in the embodiment of Figure 3 above, and will not be repeated here.

[0243] The region mapping module 13 includes: a reference pixel point determination unit 131, a reference pixel position determination unit 132, a mapping pixel position determination unit 133, and a mapping region determination unit 134;

[0244] The reference pixel determination unit 131 is configured to determine P reference pixels in the local reference area; P is a positive integer greater than 1;

[0245] A reference pixel position determining unit 132 is configured to determine pixel positions of P reference pixels in a target coordinate system;

[0246] A mapping pixel position determining unit 133 is configured to determine, based on the pixel position transformation relationship and the pixel positions of the P reference pixels, the P mapping pixels corresponding to the P reference pixels in the service image; one reference pixel corresponds to one mapping pixel;

[0247] The mapping area determining unit 134 is configured to determine a local mapping area corresponding to the local reference area in the service image based on the P mapping pixels.

[0248] Among them, the processing procedures of the reference pixel point determination unit 131, the reference pixel position determination unit 132, the mapping pixel position determination unit 133, and the mapping area determination unit 134 can refer to the relevant description in the embodiment of Figure 3 above, and will not be repeated here.

[0249] The curve fitting module 14 includes: a network acquisition unit 141, a probability calculation unit 142, and a foreground determination unit 143;

[0250] A network acquisition unit 141 is used to acquire a target foreground recognition network for foreground area recognition;

[0251] The probability calculation unit 142 is configured to input the target positioning image corresponding to the target positioning area into the target foreground recognition network, which performs foreground area recognition processing on the target positioning image to obtain the foreground probability of each pixel in the target positioning image; the foreground probability refers to the probability that each pixel identified in the target positioning image belongs to the foreground area;

[0252] The foreground determination unit 143 is configured to determine a foreground area corresponding to the target object based on the foreground probability of each pixel in the target positioning image.

[0253] The processing procedures of the network acquisition unit 141 , the probability calculation unit 142 , and the foreground determination unit 143 may refer to the relevant description in the embodiment of FIG. 3 , and are not described in detail here.

[0254] The curve fitting module 14 further includes: a network training unit 144;

[0255] The network training unit 144 is specifically configured to:

[0256] Acquire a sample object image; the sample object image is associated with sample foreground region annotation information of the sample object;

[0257] Obtaining an initial foreground recognition network, inputting the sample object image into the initial foreground recognition network, and having the initial foreground recognition network perform foreground region recognition processing on the sample object image to obtain a foreground probability for each pixel in the sample object image;

[0258] Based on the foreground probability of each pixel in the sample object image and the sample foreground area labeling information, the initial foreground recognition network is iteratively trained, and the initial foreground recognition network after iterative training is determined as the target foreground recognition network.

[0259] The processing process of the network training unit 144 may refer to the relevant description in the embodiment of FIG3 above, and will not be described in detail here.

[0260] The curve fitting module 14 includes: an edge pixel determination unit 145, a fitting pixel determination unit 146, a fitting curve determination unit 147, and an edge curve determination unit 148;

[0261] The edge pixel determination unit 145 is configured to determine an edge pixel set of the target object from the pixels in the foreground area; the edge pixel set includes N edge pixel points; N is a positive integer;

[0262] The fitting pixel determination unit 146 is configured to select M edge pixel points from the N edge pixel points included in the edge pixel point set, and determine the selected M edge pixel points as M first fitting pixel points; M is a positive integer less than N;

[0263] A fitting curve determining unit 147 is configured to perform curve fitting processing based on the M first fitting pixel points to obtain a fitting curve corresponding to the M first fitting pixel points;

[0264] The edge curve determining unit 148 is configured to determine the edge curve of the target object based on the fitting curves corresponding to the M first fitting pixel points.

[0265] The processing procedures of the edge pixel determination unit 145 , the fitting pixel determination unit 146 , the fitting curve determination unit 147 , and the edge curve determination unit 148 may refer to the relevant description in the embodiment of FIG. 12 , and are not described in detail here.

[0266] The edge curve determining unit 148 includes: a distance determining unit 1481, an inner group pixel point determining unit 1482, an inner group pixel number determining unit 1483, and a number comparing unit 1484;

[0267] The distance determining unit 1481 is configured to determine the fitting curve corresponding to the M first fitting pixel points as the first fitting curve, and calculate the distance between each edge pixel point of the N edge pixel points and the first fitting curve;

[0268] An in-group pixel point determining unit 1482 is configured to determine, based on a distance to the first fitting curve, an in-group pixel point of the first fitting curve from the N edge pixel points; an in-group pixel point is an edge pixel point whose distance to the fitting curve is less than or equal to a distance threshold;

[0269] The inner group pixel number determining unit 1483 is configured to count the number of pixels of the inner group pixel points associated with the first fitting curve, and determine the counted number of pixels as the inner group pixel number of the first fitting curve;

[0270] The quantity comparison unit 1484 is configured to determine the first fitting curve as an edge curve of the target object if the number of pixels in the inner group of the first fitting curve is greater than or equal to a quantity threshold.

[0271] Among them, the processing procedures of the distance determination unit 1481, the inner group pixel point determination unit 1482, the inner group pixel number determination unit 1483, and the number comparison unit 1484 can refer to the relevant description in the embodiment of Figure 12 above, and will not be repeated here.

[0272] The edge curve determination unit 148 includes: a fitting number recording unit 1485, a number comparison unit 1486, a target number determination unit 1487, and a curve determination unit 1488;

[0273] The fitting number recording unit 1485 is configured to record the number of curve fittings associated with the edge curve when determining the fitting curve corresponding to the M first fitting pixel points, and determine the number of inner group pixels of the fitting curve corresponding to the M first fitting pixel points as the initial inner group pixel number; the inner group pixel number is the number of pixels of the inner group pixel points of the fitting curve;

[0274] The number comparison unit 1486 is configured to notify the fitting pixel determination unit 146 to execute the step of selecting M edge pixel points from the N edge pixel points included in the edge pixel point set when the number of curve fitting times does not reach the fitting number threshold;

[0275] The fitting curve determining unit 147 is configured to determine the selected M edge pixels as M second fitting pixels, perform curve fitting processing based on the M second fitting pixels, and obtain fitting curves corresponding to the M second fitting pixels;

[0276] The inner group pixel number determining unit 1483 is configured to obtain the inner group pixel number of the fitting curve corresponding to the M second fitting pixel points, and determine the inner group pixel number of the fitting curve corresponding to the M second fitting pixel points as the inner group pixel number to be compared;

[0277] The target number determining unit 1487 is configured to determine a target number of inner group pixels based on the number of inner group pixels to be compared and the initial number of inner group pixels; the target number of inner group pixels is the largest number of inner group pixels between the number of inner group pixels to be compared and the initial number of inner group pixels;

[0278] The curve determining unit 1488 is configured to determine the edge curve of the target object based on the fitting curve corresponding to the number of pixels in the target group.

[0279] Among them, the processing procedures of the fitting number recording unit 1485, the number comparison unit 1486, the target number determination unit 1487, and the curve determination unit 1488 can refer to the relevant description in the embodiment of Figure 12 above, and will not be repeated here.

[0280] The curve determination unit 1488 is specifically configured to:

[0281] When the fitting curves corresponding to the M second fitting pixel points are determined, the number of curve fitting times is updated;

[0282] When the updated curve fitting times reaches a fitting times threshold, the fitting curve corresponding to the number of pixels in the target inner group is determined as the edge curve of the target object.

[0283] The fitting curve determining unit 147 is specifically configured to:

[0284] A curve function to be fitted is obtained, and a curve fitting process is performed based on the M first fitting pixel points and the curve function to be fitted to obtain a fitting curve corresponding to the M first fitting pixel points.

[0285] Wherein, the edge curve includes the edge straight line;

[0286] The image processing device 1 further includes: a detection module 15 and a management module 16;

[0287] A detection module 15 is configured to determine a pixel to be detected in the target positioning image, and determine a detection value associated with the pixel to be detected based on a distance from the pixel to be detected to an edge line;

[0288] The management module 16 is configured to perform object quality management on the target object based on the detection value.

[0289] The processing procedures of the detection module 15 and the management module 16 may refer to the relevant description in the embodiment of FIG3 , and will not be described in detail here.

[0290] Please refer to Figure 14, which is a structural diagram of a computer device provided in an embodiment of the present application. As shown in Figure 14, the computer device 1000 may include: a processor 1001, a network interface 1004 and a memory 1005. In addition, the above-mentioned computer device 1000 may also include: a user interface 1003, and at least one communication bus 1002. Among them, the communication bus 1002 is used to realize the connection and communication between these components. Among them, the user interface 1003 may include a display screen (Display), a keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface). The memory 1005 may be a high-speed RAM memory or a non-volatile memory (non-volatile memory), such as at least one disk storage. The memory 1005 may also be at least one storage device located away from the aforementioned processor 1001. As shown in Figure 14, the memory 1005 as a computer-readable storage medium may include an operating system, a network communication module, a user interface module and a device control application.

[0291] In the computer device 1000 shown in FIG14 , the network interface 1004 can provide network communication functions; the user interface 1003 is primarily used to provide an interface for user input; and the processor 1001 can be used to invoke the device control application stored in the memory 1005 to execute the description of the image processing method in any of the corresponding embodiments described above, which will not be repeated here. Furthermore, the description of the beneficial effects of using the same method will not be repeated here.

[0292] In addition, it should be pointed out here that: the embodiment of the present application also provides a computer-readable storage medium, and the computer-readable storage medium stores a computer program executed by the image processing device mentioned above, and the computer program includes program instructions. When the processor executes the program instructions, it can execute the description of the image processing method in the above embodiment. Therefore, it will not be repeated here. In addition, the description of the beneficial effects of using the same method will not be repeated. For technical details not disclosed in the computer-readable storage medium embodiment involved in this application, please refer to the description of the method embodiment of this application.

[0293] The computer-readable storage medium may be an internal storage unit of the image processing apparatus or the computer device provided in any of the aforementioned embodiments, such as a hard disk or memory of the computer device. The computer-readable storage medium may also be an external storage device of the computer device, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the computer device. Furthermore, the computer-readable storage medium may also include both an internal storage unit of the computer device and an external storage device. The computer-readable storage medium is used to store the computer program and other programs and data required by the computer device. The computer-readable storage medium may also be used to temporarily store data that has been output or is to be output.

[0294] In addition, it should be noted that the embodiments of the present application also provide a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the method provided by any of the corresponding embodiments above. In addition, the description of the beneficial effects of using the same method will not be repeated. For technical details not disclosed in the computer program product or computer program embodiments involved in this application, please refer to the description of the method embodiments of this application.

[0295] The terms "first", "second", etc. in the description, claims, and drawings of the embodiments of the present application are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "comprising" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, apparatus, product, or device comprising a series of steps or units is not limited to the listed steps or modules, but may optionally include steps or modules not listed, or may optionally include other step units inherent to these processes, methods, apparatuses, products, or devices.

[0296] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0297] The above disclosure is only a preferred embodiment of the present application, and certainly cannot be used to limit the scope of rights of the present application. Therefore, equivalent changes made according to the claims of the present application are still within the scope covered by the present application.

Claims

1. An image processing method, characterized in that: The method comprises: Acquire a business image containing a target object, acquire a template image for assisting in positioning the target object, and determine an object detection area corresponding to the template object from the template image; the template object refers to an object existing in the template image and having the same object type as the target object, and the area size of the object detection area is smaller than the area size of the business image; Searching for an area matching the object detection area in the business image, determining the found area as an initial positioning area of ​​the target object, and determining a pixel position transformation relationship based on pixel positions of pixels in the object detection area and pixel positions of pixels in the initial positioning area; When determining the local reference area from the template image, a local mapping area corresponding to the local reference area is determined in the business image based on the pixel position transformation relationship, and the local mapping area is used as the target positioning area of ​​the target object; the local mapping area refers to the area obtained after pixel positions of pixels in the local reference area are transformed based on the pixel position transformation relationship; A foreground area corresponding to the target object is identified in the target positioning image corresponding to the target positioning area, and edge curve fitting processing is performed on pixel points in the foreground area to obtain an edge curve of the target object.

2. The method according to claim 1, characterized in that The searching for an area matching the object detection area in the business image includes: Based on the area size of the object detection area, a sliding window for window sliding is determined in the business image, and a window position of the sliding window is determined in the business image as a to-be-matched area in the business image; the area size of the to-be-matched area is consistent with the area size of the object detection area; Determine the area similarity between the area to be matched and the object detection area by using the pixel value of each pixel point in the area to be matched and the pixel value of each pixel point in the object detection area; Based on the region similarity, a region matching the object detection region is determined in the business image.

3. The method according to any one of claims 1 to 2, characterized in that: The to-be-matched area includes a first sliding window area and a second sliding window area; the first sliding window area is an area determined when the sliding window slides to a first window position; the second sliding window area is an area determined when the sliding window slides to a second window position; the second window position is a window position next to the first window position; The determining the region similarity between the region to be matched and the object detection region by using the pixel value of each pixel point in the region to be matched and the pixel value of each pixel point in the object detection region comprises: Determine the regional similarity between the first sliding window area and the object detection area by using the pixel value of each pixel point in the first sliding window area and the pixel value of each pixel point in the object detection area, and determine the regional similarity between the first sliding window area and the object detection area as the first regional similarity; Determine the regional similarity between the second sliding window area and the object detection area by using the pixel value of each pixel point in the second sliding window area and the pixel value of each pixel point in the object detection area, and determine the regional similarity between the second sliding window area and the object detection area as the second regional similarity; The region similarity between the to-be-matched region and the object detection region is determined based on the first region similarity and the second region similarity.

4. The method according to any one of claims 1 to 3, characterized in that: The determining, based on the region similarity, a region matching the object detection region in the business image comprises: Determine a target region similarity based on the first region similarity and the second region similarity, the target region similarity being a region similarity with a maximum value determined between the first region similarity and the second region similarity; In the business image, an area matching the object detection area is determined based on the to-be-matched area corresponding to the target area similarity.

5. The method according to any one of claims 1 to 4, characterized in that: The first sliding window area includes an i-th pixel point, and the object detection area includes a j-th pixel point; i and j are consistent positive integers, and the pixel position of the i-th pixel point in the first sliding window area is consistent with the pixel position of the j-th pixel point in the object detection area; The determining the regional similarity between the first sliding window area and the object detection area by using the pixel value of each pixel point in the first sliding window area and the pixel value of each pixel point in the object detection area includes: An average value obtained by performing mean calculation on the pixel values ​​of the pixel points in the first sliding window area is determined as a first pixel mean value of the first sliding window area; Determine an average value obtained by performing mean calculation on pixel values ​​of pixel points in the object detection area as a second pixel mean value of the object detection area; Determine the difference between the pixel value of the i-th pixel point in the first sliding window area and the first pixel mean as the first difference value corresponding to the i-th pixel point, and determine the difference between the pixel value of the j-th pixel point in the object detection area and the second pixel mean as the second difference value corresponding to the j-th pixel point; Based on the first difference value corresponding to the i-th pixel point and the second difference value corresponding to the j-th pixel point, the regional similarity between the first sliding window area and the object detection area is determined.

6. The method according to any one of claims 1 to 5, characterized in that: The determining of the pixel position transformation relationship based on the pixel positions of the pixel points in the object detection area and the pixel positions of the pixel points in the initial positioning area includes: Determine K key pixel points from the initial positioning area; K is a positive integer greater than 1; Determine K associated pixel points corresponding to the K key pixel points in the object detection area; one key pixel point corresponds to one associated pixel point; Based on the K key pixel points and the K associated pixel points, K pixel point pairs are constructed; a pixel point pair includes a key pixel point and a corresponding associated pixel point; In the target coordinate system, determining the pixel positions of the K key pixels and the pixel positions of the K associated pixels; Based on the pixel positions of the key pixels and the pixel positions of the associated pixels included in each of the K pixel pairs, a pixel position transformation matrix for representing the pixel position transformation relationship is calculated.

7. The method according to any one of claims 1 to 6, characterized in that: The determining, based on the pixel position transformation relationship, a local mapping area corresponding to the local reference area in the business image, and using the local mapping area as a target positioning area of ​​the target object includes: Determine P reference pixel points in the local reference area; P is a positive integer greater than 1; In the target coordinate system, determining the pixel positions of the P reference pixels; Based on the pixel position transformation relationship and the pixel positions of the P reference pixels, determining P mapping pixels corresponding to the P reference pixels in the business image; one reference pixel corresponds to one mapping pixel; A local mapping area corresponding to the local reference area is determined in the business image based on the P mapping pixels.

8. The method according to any one of claims 1 to 7, characterized in that: The step of identifying a foreground area corresponding to the target object in the target positioning image corresponding to the target positioning area includes: Obtaining a target foreground recognition network for performing foreground region recognition; The target positioning image corresponding to the target positioning area is input into the target foreground recognition network, and the target foreground recognition network performs foreground area recognition processing on the target positioning image to obtain the foreground probability of each pixel in the target positioning image; the foreground probability refers to the probability that each pixel identified in the target positioning image belongs to the foreground area; The foreground area corresponding to the target object is determined based on the foreground probability of each pixel in the target positioning image.

9. The method according to any one of claims 1 to 8, characterized in that: The method further comprises: Acquire a sample object image; the sample object image is associated with sample foreground area annotation information of the sample object; Obtaining an initial foreground recognition network, inputting the sample object image into the initial foreground recognition network, and having the initial foreground recognition network perform foreground region recognition processing on the sample object image to obtain a foreground probability of each pixel in the sample object image; Based on the foreground probability of each pixel in the sample object image and the sample foreground region labeling information, the initial foreground recognition network is iteratively trained, and the initial foreground recognition network after iterative training is determined as the target foreground recognition network.

10. The method according to any one of claims 1 to 9, characterized in that: The performing edge curve fitting processing on the pixel points in the foreground area to obtain the edge curve of the target object includes: Determine an edge pixel point set of the target object from the pixel points in the foreground area; the edge pixel point set includes N edge pixel points; N is a positive integer; Selecting M edge pixel points from the N edge pixel points included in the edge pixel point set, and determining the selected M edge pixel points as M first fitting pixel points; M is a positive integer less than N; Performing curve fitting processing based on the M first fitting pixel points to obtain a fitting curve corresponding to the M first fitting pixel points; An edge curve of the target object is determined based on the fitting curves corresponding to the M first fitting pixels.

11. The method according to any one of claims 1 to 10, characterized in that: The step of determining the edge curve of the target object based on the fitting curve corresponding to the M first fitting pixel points includes: Determine the fitting curve corresponding to the M first fitting pixel points as a first fitting curve, and calculate the distance from each edge pixel point of the N edge pixel points to the first fitting curve; Based on the distance to the first fitting curve, determining an inner group pixel point of the first fitting curve from the N edge pixel points; the inner group pixel point refers to an edge pixel point whose distance to the fitting curve is less than or equal to a distance threshold; Counting the number of pixels of the inner group pixel points associated with the first fitting curve, and determining the counted number of pixels as the number of pixels of the inner group of the first fitting curve; If the number of pixels in the inner group of the first fitting curve is greater than or equal to the number threshold, the first fitting curve is determined as the edge curve of the target object.

12. The method according to any one of claims 1 to 11, characterized in that: The step of determining the edge curve of the target object based on the fitting curve corresponding to the M first fitting pixel points includes: When determining the fitting curve corresponding to the M first fitting pixel points, the number of curve fittings associated with the edge curve is recorded, and the number of inner group pixels of the fitting curve corresponding to the M first fitting pixel points is determined as the initial inner group pixel number; the inner group pixel number is the number of pixels of the inner group pixel points of the fitting curve; When the number of curve fitting times does not reach the fitting times threshold, notifying to execute the step of selecting M edge pixel points from the N edge pixel points included in the edge pixel point set; Determine the selected M edge pixel points as M second fitting pixel points, perform curve fitting processing based on the M second fitting pixel points, and obtain fitting curves corresponding to the M second fitting pixel points; Acquire the number of inner group pixels of the fitting curve corresponding to the M second fitting pixel points, and determine the number of inner group pixels of the fitting curve corresponding to the M second fitting pixel points as the number of inner group pixels to be compared; Based on the number of inner group pixels to be compared and the initial number of inner group pixels, determining a target number of inner group pixels; the target number of inner group pixels is the largest number of inner group pixels between the number of inner group pixels to be compared and the initial number of inner group pixels; The edge curve of the target object is determined based on the fitting curve corresponding to the number of pixels in the target group.

13. The method according to any one of claims 1 to 12, characterized in that: The step of determining the edge curve of the target object based on the fitting curve corresponding to the number of pixels in the target group includes: When the fitting curves corresponding to the M second fitting pixel points are determined, updating the number of curve fitting times; When the updated curve fitting times reaches the fitting times threshold, the fitting curve corresponding to the number of pixels in the target inner group is determined as the edge curve of the target object.

14. The method according to any one of claims 1 to 13, characterized in that: The performing curve fitting processing based on the M first fitting pixel points to obtain a fitting curve corresponding to the M first fitting pixel points includes: A curve function to be fitted is obtained, and a curve fitting process is performed based on the M first fitting pixel points and the curve function to be fitted to obtain a fitting curve corresponding to the M first fitting pixel points.

15. The method according to any one of claims 1 to 14, characterized in that: The edge curve includes an edge straight line; The method comprises: Determine a pixel point to be detected in the target positioning image, and determine a detection value associated with the pixel point to be detected based on a distance from the pixel point to be detected to the edge straight line; Object quality management is performed on the target object based on the detection value.

16. An image processing device, characterized in that: The device comprises: An image acquisition module, used to acquire a business image containing a target object, acquire a template image for assisting in locating the target object, and determine an object detection area from the template image; the area size of the object detection area is smaller than the area size of the business image; a transformation relationship determination module, configured to search for an area matching the object detection area in the business image, determine the found area as an initial positioning area of ​​the target object, and determine a pixel position transformation relationship based on pixel positions of pixels in the object detection area and pixel positions of pixels in the initial positioning area; A region mapping module, configured to determine a local mapping region corresponding to the local reference region in the business image based on the pixel position transformation relationship when determining the local reference region from the template image, and use the local mapping region as a target positioning region of the target object; The curve fitting module is used to identify the foreground area corresponding to the target object in the target positioning image corresponding to the target positioning area, and perform edge curve fitting processing on the pixel points in the foreground area to obtain the edge curve of the target object.

17. A computer device, characterized in that: including memory and processor; The memory is connected to the processor, the memory is used to store a computer program, and the processor is used to call the computer program so that the computer device executes the method according to any one of claims 1 to 15.

18. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the computer program is suitable for being loaded and executed by a processor, so that a computer device having the processor executes the method according to any one of claims 1 to 15.

19. A computer program product, characterized in that The method comprises a computer program / instruction, which implements the method according to any one of claims 1 to 15 when executed by a processor.

Images