Image processing method and related equipment

Abstract

The embodiment of the invention provides an image processing method and related equipment, and relates to the field of image processing. The method comprises the steps that a first camera is controlled to move in a first plane perpendicular to the orientation of the first camera, image acquisition is carried out, and a first object is recorded in a first image acquired by the first camera; according to the corresponding relation between the pixel displacement and the collection displacement, first collection displacement corresponding to the first pixel displacement is determined, and the first pixel displacement is the displacement of the first pixel position of the first object in the first image relative to the center of the first image; the first position of the first object is determined according to the first collection displacement and the first camera position where the first camera is located when the first camera collects the first image, and the corresponding relation is determined based on a plurality of detection images which are collected by the camera and record the detection object and the position of the camera when each detection image is collected. According to the scheme, the accuracy of the determined object position can be improved.

Description

Technical Field

[0001] The present application relates to the field of image processing, and in particular to an image processing method and related equipment. Background Art

[0002] By capturing images of the object, the position of the object can be detected without any contact. This contactless position detection method eliminates the need for personnel or probes to enter hazardous areas during the measurement process, creating irreplaceable advantages in fields such as precision manufacturing, life sciences, autonomous driving, hazardous environments, and large-scale logistics.

[0003] How to improve the accuracy of object positioning results is a problem that needs to be solved urgently. Summary of the Invention

[0004] The embodiments of the present application provide an image processing method and related equipment, which can improve the accuracy of positioning results.

[0005] In a first aspect, an image processing method is provided, comprising: controlling a first camera to move in a first plane perpendicular to the direction of the first camera and performing image acquisition, wherein a first object is recorded in a first image acquired by the first camera; determining a first acquisition displacement corresponding to the first pixel displacement according to a correspondence between pixel displacement and acquisition displacement, wherein the first pixel displacement is a displacement of a first pixel position of the first object in the first image relative to a center of the first image; determining a first position of the first object according to the first acquisition displacement and a first camera position at which the first camera is located when acquiring the first image; wherein the correspondence is determined according to a plurality of detection pixel displacements and a detection sampling displacement corresponding to each detection pixel displacement, wherein the correspondence is determined according to a detection data set, wherein the detection data set includes a detection pixel displacement and a detection acquisition displacement corresponding to each preset area in a plurality of preset areas, wherein In the embodiment, the detection pixel displacement corresponding to any preset area represents the displacement of the position of the detection object in the second detection image in which the detection object is located in the preset area relative to the center of the second detection image. The detection sampling displacement corresponding to any preset area represents the displacement of the position of the third camera when capturing the second detection image in which the detection object is located in the preset area relative to the first position of the third camera when capturing the first detection image. The multiple second detection images and the first detection image in which the detection object is respectively located in the multiple preset areas are all captured by the third camera during movement in a third plane. The third plane is perpendicular to the direction of the third camera. The distance between the third plane and the detection object is a first distance. The distance between the first plane and the first object is a second distance. The first distance is equal to the second distance. The third camera and the first camera are the same camera.

[0006] In some possible implementations, the correspondence is determined through a first operation, which includes: determining an initial correspondence between pixel displacement and acquisition displacement based on the detection data set; determining a predicted sampling displacement corresponding to each detection pixel displacement in the detection data set based on the initial correspondence, wherein the predicted sampling displacement corresponding to any detection pixel displacement and the any detection pixel displacement correspond to the same preset area; determining the initial correspondence as the correspondence when the difference between the detection sampling displacement corresponding to each preset area in the detection data set and the predicted sampling displacement is less than a difference threshold; and performing a second operation when there is at least one target area among the multiple preset areas, wherein the detection sampling displacement corresponding to any one of the at least one target area is the same as the predicted sampling displacement. The difference between the displacement and the predicted sampling displacement is greater than or equal to a difference threshold, and the second operation includes: for any target area, controlling the third camera to move in the third plane to a position whose displacement relative to the first position is the predicted sampling displacement corresponding to the any target area, and collecting a third detection image corresponding to the any target area; updating the detection acquisition displacement corresponding to the any target area in the detection data set to the predicted sampling displacement corresponding to the any target area, and updating the detection pixel displacement corresponding to the any target area to the displacement of the position of the detection object in the third detection image corresponding to the any target area relative to the center of the third detection image; after updating the detection data set based on each target area, determining the corresponding relationship according to the updated detection data set.

[0007] In some possible implementations, after updating the detection data set based on each target area, determining the correspondence relationship based on the updated detection data set includes: after updating the detection data set based on each target area, redetermining the initial correspondence relationship based on the updated detection data set, and re-performing the predicted sampling displacement corresponding to each detection pixel displacement until the difference between the detection sampling displacement corresponding to each second detection image in the detection data set and the predicted sampling displacement is less than the difference threshold.

[0008] In some possible implementations, the multiple preset areas include other areas among the multiple areas obtained by evenly dividing the image captured by the third camera, except for the area where the center of the image is located.

[0009] In some possible implementations, the corresponding relationship includes a first proportional relationship between the pixel displacement and the acquisition displacement along a first direction, and a second proportional relationship between the pixel displacement and the acquisition displacement along a second direction, wherein the first direction is perpendicular to the second direction.

[0010] In some possible implementations, determining the first position of the first object based on the first acquisition displacement includes: controlling the first camera to move according to the first acquisition displacement and acquiring a first verification image; determining, based on the correspondence, a verification acquisition displacement corresponding to a verification pixel displacement, where the verification pixel displacement is the displacement of the verification pixel position of the first object in the verification image relative to the center of the verification image; and determining the first position based on the verification acquisition displacement.

[0011] In some possible implementations, the method further includes: controlling a second camera to move in the first plane so that a second object is recorded in a second image captured by the second camera, the orientation of the second camera is the same as that of the first camera, and a third distance between the second object and the first plane is equal to the first distance; determining a second acquisition displacement corresponding to the second pixel displacement based on a correspondence between pixel displacement and acquisition displacement, where the second pixel displacement is a displacement of a second pixel position of the second object in the second image relative to a center of the first image; determining the first position of the first object based on the first acquisition displacement and a first camera position at which the first camera captured the first image, includes: determining a relative positional relationship between the first object and the second object based on the first acquisition displacement, the second acquisition displacement, the first camera position, and the second camera position, where the second camera position is the position of the second camera at which the second camera captured the second image.

[0012] In a second aspect, an embodiment of the present application provides an image processing method, comprising: controlling a third camera to move in a third plane perpendicular to the direction of the third camera, and controlling the third camera to capture a first detection image and multiple second detection images, wherein the detection object recorded in the first detection image is located at the center of the first detection image, and the detection object in different second detection images is located in different preset areas; determining the correspondence between pixel displacement and acquisition displacement according to a detection data set, the detection data set including the detection pixel displacement and detection acquisition displacement corresponding to each of the multiple preset areas, wherein the detection pixel displacement corresponding to any preset area represents the displacement of the position of the detection object in the second detection image where the detection object is located in any preset area relative to the center of the second detection image, and the detection acquisition displacement corresponding to any preset area represents the acquisition of the detection object located in the The detection acquisition displacement of the position of the third camera when capturing the second detection image of any preset area relative to the first position of the third camera when capturing the first detection image; the corresponding relationship is used to determine the first acquisition displacement corresponding to the first pixel displacement, the first pixel displacement being the displacement of the first pixel position of the first object in the first image captured by the first camera relative to the center of the first image, the first image being captured by the first camera in the process of moving in a first plane perpendicular to the direction of the first camera, the first acquisition displacement and the position of the first camera when capturing the first image are used to determine the first position of the first object, the distance between the third plane and the detection object is the first distance, the distance between the first plane and the first object is the second distance, the first distance is equal to the second distance, and the third camera and the first camera are the same camera.

[0013] In some possible implementations, determining the correspondence between pixel displacement and acquisition displacement based on the detection data set includes: determining an initial correspondence between pixel displacement and acquisition displacement based on the detection data set; determining a predicted sampling displacement corresponding to each detection pixel displacement in the detection data set based on the initial correspondence, wherein the predicted sampling displacement corresponding to any detection pixel displacement and the any detection pixel displacement correspond to the same preset area; determining the initial correspondence as the correspondence when the difference between the detection sampling displacement corresponding to each preset area in the detection data set and the predicted sampling displacement is less than a difference threshold; and performing a second operation when there is at least one target area among the multiple preset areas, wherein the detection sampling displacement corresponding to any one of the at least one target area is the same as the predicted sampling displacement. The difference between the sampling displacement and the predicted sampling displacement is greater than or equal to a difference threshold, and the second operation includes: for any target area, controlling the third camera to move in the third plane to a position whose displacement relative to the first position is the predicted sampling displacement corresponding to the any target area, and collecting a third detection image corresponding to the any target area; updating the detection acquisition displacement corresponding to the any target area in the detection data set to the predicted sampling displacement corresponding to the any target area, and updating the detection pixel displacement corresponding to the any target area to the displacement of the position of the detection object in the third detection image corresponding to the any target area relative to the center of the third detection image; after updating the detection data set based on each target area, determining the corresponding relationship according to the updated detection data set.

[0014] In a third aspect, an embodiment of the present application provides an image processing device, comprising a unit for executing the method of the first aspect or the second aspect.

[0015] In a fourth aspect, an embodiment of the present application provides an electronic device comprising a processor and a memory, wherein the memory is used to store a computer program, and the processor is used to call and run the computer program from the memory, so that the electronic device executes the method of the first aspect or the second aspect.

[0016] In a fifth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer storage medium stores a computer program, and when the computer program is executed, the method of the first aspect or the second aspect mentioned above is executed.

[0017] In a sixth aspect, an embodiment of the present application provides a computer program product, which includes computer program instructions. When the computer program instructions are executed, the method of the first aspect or the second aspect mentioned above is executed.

[0018] In the seventh aspect, an embodiment of the present application provides a chip, including a processor and a data interface, wherein the processor reads instructions stored in a memory through the data interface to implement the method of the first or second aspect above.

[0019] In an eighth aspect, an embodiment of the present application provides an image processing device, comprising a first sensor and a processor. The first sensor is configured to acquire a first image and first pose information; and the processor is configured to execute the image processing method described in the first aspect.

[0020] The beneficial effects of the embodiments of the present application compared with the prior art are: the camera captures an image of the detection object on a plane perpendicular to the direction of the camera, and the correspondence between the pixel displacement and the capture displacement is determined based on the displacement of the position of the camera capturing the image relative to the camera position that makes the detection object located at the center of the image, and the displacement of the position of the detection object in the image captured by the camera relative to the center of the image. Therefore, based on the correspondence, the first capture displacement corresponding to the first pixel displacement of the position of the first object in the image captured by the camera relative to the center of the image is determined, and the position of the first object is determined based on the first capture displacement and the position of the camera when capturing the image of the first object, thereby improving the accuracy of the determined object position. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 is a schematic flow chart of an image processing method provided in an embodiment of the present application; Figure 2 Schematic diagram of image acquisition by a camera provided in an embodiment of the present application; Figure 3 is a schematic flow chart of an image processing method provided in an embodiment of the present application; Figure 4 is a schematic flow chart of an image processing method provided in an embodiment of the present application; Figure 5 is a schematic structural diagram of an image processing device provided in an embodiment of the present application; Figure 6 This is a structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0022] The technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application.

[0023] In the following description, specific details such as specific system structures and techniques are provided for purposes of illustration rather than limitation to facilitate a thorough understanding of the embodiments of the present application. However, it will be apparent to those skilled in the art that the present application may be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid obscuring the description of the present application with unnecessary detail.

[0024] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, integers, steps, operations, elements and / or components, but does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or collections thereof.

[0025] It will also be understood that the term "and / or" used in this specification and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

[0026] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0027] References to "one embodiment" or "some embodiments" in this specification mean that a particular feature, structure, or characteristic described in conjunction with the embodiment is included in one or more embodiments of the present application. Thus, phrases such as "in one embodiment," "in some embodiments," "in other embodiments," and "in yet other embodiments" appearing in various places in this specification do not necessarily refer to the same embodiment, but rather mean "one or more but not all embodiments," unless otherwise specifically emphasized.

[0028] By acquiring images, the relative position of multiple objects can be detected without contact. This contactless phase position measurement method eliminates the need for personnel or probes to enter hazardous areas, creating an irreplaceable advantage in areas such as precision manufacturing, life sciences, autonomous driving, hazardous environments, and large-scale logistics.

[0029] How to improve the accuracy of measurement results is an urgent problem to be solved.

[0030] In view of this, the present application provides a related image processing solution that can improve the accuracy of the detection results. The solution provided by the present application is described below.

[0031] Figure 1 This is a schematic flowchart of an image processing method provided in an embodiment of the present application. Figure 1 The method shown includes steps S101 to S105.

[0032] Step S101, control the third camera to move in a third plane perpendicular to the direction of the third camera, and control the third camera to capture a first detection image and multiple second detection images, wherein the detection object recorded in the first detection image is located at the center of the first detection image, and the detection objects in different second detection images are located in different preset areas.

[0033] The first detection image and the plurality of second detection images are both images captured by the third camera, and have the same size as the plurality of second detection images.

[0034] During the movement of the third camera in the third plane, the relative position relationship between the third camera and the detection object changes, so that at different time points, the position of the detection object in the image captured by the third camera may be different.

[0035] The number of the plurality of second detection images may be preset. Figure 2 As shown, in different second detection images captured by the third camera, the detection object is located in different preset areas. There may be no overlap between two adjacent preset areas. The multiple preset areas can cover the entire image captured by the third camera.

[0036] The image captured by the third camera may be rectangular. The image captured by the third camera is divided into a plurality of regions. The plurality of preset regions may be the plurality of divided regions. Alternatively, the plurality of preset regions may be a plurality of regions other than the region where the image center is located within the plurality of divided regions. The plurality of preset regions may be of equal or unequal sizes.

[0037] That is, the plurality of preset areas include areas other than the area where the center of the image is located, among the plurality of areas obtained by evenly dividing the image captured by the third camera. The plurality of preset areas may also include the area where the center of the image is located.

[0038] For example, Figure 2 As shown, the image captured by the third camera can be evenly divided into nine equal-sized areas in three rows and three columns according to a nine-square grid. The nine areas can each be nine uniform rectangles. Each rectangle can serve as a preset area. Alternatively, the other eight rectangular areas, excluding the area where the center point of the image is located, can each serve as a preset area.

[0039] Step S102: Determine a correspondence between a pixel displacement and an acquisition displacement based on a detection data set, wherein the detection data set includes a detection pixel displacement and a detection acquisition displacement corresponding to each of the multiple preset areas, wherein the detection pixel displacement corresponding to any preset area represents a displacement of a position of the detection object in a second detection image in which the detection object is located in the preset area relative to a center of the second detection image, and the detection acquisition displacement corresponding to any preset area represents a detection acquisition displacement of a position of the third camera when capturing the second detection image in which the detection object is located in the preset area relative to a first position of the third camera when capturing the first detection image.

[0040] Based on the detection data set, the initial correspondence between pixel displacement and acquisition displacement can be determined. The initial correspondence can be used as the correspondence between pixel displacement and acquisition displacement for Figure 4 The image processing method shown.

[0041] Alternatively, after determining the initial correspondence, the predicted sampling displacement corresponding to each detection pixel displacement in the detection data set may be determined based on the initial correspondence. The predicted sampling displacement corresponding to any detection pixel displacement corresponds to the same preset area as the detection pixel displacement.

[0042] When the difference between the detection sampling displacement and the predicted sampling displacement corresponding to each preset area in the detection data set is less than the difference threshold, the initial correspondence relationship is determined to be Figure 4 The correspondence used in the image processing method shown.

[0043] When the difference between the detection sampling displacement and the predicted sampling displacement corresponding to each preset area in the detection data set is not less than the difference threshold, image acquisition can be performed again through the third camera based on the predicted sampling displacement, and the initial correspondence relationship can be re-determined.

[0044] If the difference between the detected sampling displacement and the predicted sampling displacement corresponding to each preset area in the detection data set is not less than the difference threshold, the multiple preset areas may include at least one target area, and the difference between the detected sampling displacement and the predicted sampling displacement corresponding to any target area is greater than or equal to the difference threshold.

[0045] For any target area, the third camera can be controlled to move in the third plane to a position where the displacement relative to the first position is the predicted sampling displacement corresponding to the any target area, and a third detection image corresponding to the any target area is collected.

[0046] Afterwards, the detection acquisition displacement corresponding to any target area in the detection data set is updated to the predicted sampling displacement corresponding to any target area, and the detection pixel displacement corresponding to any target area is updated to the displacement of the position of the detection object in the third detection image corresponding to any target area relative to the center of the third detection image.

[0047] After the detection data set is updated based on each target region, the initial correspondence relationship may be re-determined according to the updated detection data set.

[0048] It should be understood that by reasonably controlling the position of the third camera so that in each second detection image, the distance between the detection object and the edge of the preset area is greater than the preset value, the predicted sampling displacement corresponding to any preset area determined based on the initial correspondence can be located in any preset area.

[0049] For example, if the position represented by the predicted sampling displacement corresponding to each target area is within the target area, a third detection image corresponding to each target area can be captured, and the initial correspondence relationship can be re-determined using the predicted third detection image. If the position represented by the predicted sampling displacement corresponding to a target area is outside the target area, a second detection image can be re-captured for each preset area, and the initial correspondence relationship can be re-determined based on the re-captured second detection image.

[0050] The initial photographing position, i.e., the position at which the third camera captures the second detection image, can be manually set or randomly determined. That is, the values ​​are scattered and not systematically related. The predicted sampling displacement corresponding to each area is an ideal value calculated based on the sampling results of each preset area. There is a systematic relationship between the predicted sampling displacements corresponding to each area. Therefore, the third detection image is captured at a position that is the predicted sampling displacement corresponding to the target area relative to the first position. This makes the capture position of the third detection image more reasonable, and thus the relationship between the detection capture displacement and the detection pixel displacement of the third detection image more reasonable.

[0051] After the initial corresponding relationship is re-determined based on the updated detection data set, the re-determined initial corresponding relationship can be used as the corresponding relationship for Figure 4 Alternatively, after each initial correspondence is determined, a predicted sampling displacement corresponding to each detection pixel displacement in the detection data set is determined based on the initial correspondence until the difference between the detection sampling displacement corresponding to each preset area in the detection data set and the predicted sampling displacement is less than a difference threshold.

[0052] Next, combine Figure 3 , Figure 1 The image processing method shown is described in detail.

[0053] Figure 3 This is a schematic flowchart of an image processing method provided in an embodiment of the present application. Figure 3 The method shown includes steps S311 to S325.

[0054] Step S311 : Control the third camera to move to a position pc0 directly above the detection object on the first plane, and capture a first detection image.

[0055] Therefore, the detection object in the first detection image is located at the center Pp0 of the first detection image.

[0056] Before step S311 is performed, i is 1. That is, the initial value of i is 1.

[0057] Step S312: Control the third camera to move in the first plane and perform image acquisition to obtain an i-th second detection image, in which the detection object is located in an i-th preset area.

[0058] When the third camera moves in the first plane, the direction of the third camera remains perpendicular to the first plane.

[0059] In the image captured by the third camera, it can be detected whether the object is located in the i-th preset area.

[0060] When the detection object is not located in the i-th preset area, the third camera may be controlled to move in the first plane and capture images again until the detection object is located in the i-th preset area in the image captured by the third camera.

[0061] Step S313 , determining the acquisition displacement ΔPci of the actual acquisition position Pci where the third sensor acquires the i-th second detection image relative to the position Pc0 when acquiring the first detection image, based on the actual acquisition position Pci of the third sensor when acquiring the i-th second detection image.

[0062] The actual acquisition position Pci of the third sensor during the i-th second detection image can be expressed as coordinates in a plane rectangular coordinate system located on the third plane. For example, with the position Pc0 of the third sensor during the first detection image acquisition as the origin, the actual acquisition position Pci of the third sensor during the i-th second detection image can be expressed as an acquisition displacement ΔPci of the actual acquisition position Pci relative to the position Pc0.

[0063] Step S314 , determining the pixel displacement ΔPpi of the pixel position Ppi of the detection object in the second detection image relative to the center Pp0 of the second detection image according to the pixel position Ppi of the detection object in the i-th second detection image.

[0064] The pixel position Ppi of the detection object in the i-th second detection image can be expressed as coordinates in the plane rectangular coordinate system of the second detection image. For example, with the center Pp0 of the second detection image as the origin, the pixel position Ppi of the detection object in the i-th second detection image can be expressed as the pixel displacement ΔPpi of the pixel position Ppi relative to the position Pp0.

[0065] Step S315 : Add the pixel displacement ΔPpi and the acquisition displacement ΔPci to the data set.

[0066] Step S316: determine whether i is greater than or equal to the number N of preset areas.

[0067] If i is less than the number of preset areas, step S319 is performed. If i is greater than or equal to the number of preset areas, step S318 may be performed.

[0068] Step S317, add 1 to i.

[0069] That is, in step S317, i=i+1 is set.

[0070] After step S317, step S312 is performed again.

[0071] Step S318: determining an initial correspondence between pixel displacement and acquisition displacement according to the data set.

[0072] That is, according to the N acquisition displacements ΔPci and the pixel displacement ΔPpi corresponding to each acquisition displacement ΔPci, the initial corresponding relationship between the pixel displacement and the acquisition displacement can be determined.

[0073] The initial correspondence between the pixel displacement and the acquisition displacement can be expressed as a first proportional relationship between the pixel displacement and the acquisition displacement in a first direction and a second proportional relationship between the pixel displacement and the acquisition displacement in a second direction. The first direction and the second direction can be perpendicular to each other. The proportional coefficients in the first proportional relationship and the second proportional relationship can be the same or different.

[0074] When the proportional coefficients in the first proportional relationship and the second proportional relationship are different, the initial corresponding relationship can be represented by a coefficient matrix. The coefficient matrix includes the proportional coefficients in the first proportional relationship and the proportional coefficients in the second proportional relationship.

[0075] For example, both the acquisition displacement ΔPci and the pixel displacement ΔPpi can be represented by coordinates. The horizontal and vertical axes of the coordinate system containing the acquisition displacement ΔPci and the pixel displacement ΔPpi can be parallel. The first direction and the second direction can be the horizontal and vertical axes of the coordinate system, respectively.

[0076] After step S318, i=1 may be set and step S319 may be performed.

[0077] Step S319 : determining the predicted acquisition displacement ΔPci′ corresponding to the pixel position Ppi where the detection object is located in the i-th second detection image according to the initial correspondence between the pixel displacement and the acquisition displacement.

[0078] Step S320 : determining whether the difference between the predicted acquisition displacement ΔPci′ and the acquisition displacement ΔPci is greater than or equal to a difference threshold.

[0079] In a case where the difference between the predicted acquisition displacement ΔPci′ and the acquisition displacement ΔPci is smaller than the difference threshold, step S323 may be performed.

[0080] If it is determined in step S320 that the difference between the predicted acquisition displacement ΔPci′ and the acquisition displacement ΔPci is greater than or equal to the difference threshold, steps S321 to S323 may be performed.

[0081] Step S321 : deleting the acquisition displacement ΔPci and the pixel displacement ΔPpi corresponding to the acquisition displacement ΔPci from the data set.

[0082] Step S322: Control the third camera to move in the third plane to a position where the displacement relative to the position Pc0 when the third camera captured the first detection image is the predicted acquisition displacement ΔPci', recapture the i-th second detection image, and record the predicted acquisition displacement ΔPci' as the acquisition displacement ΔPci.

[0083] Step S323: Determine whether i is greater than or equal to the number N of preset areas.

[0084] When i is less than N, step S324 may be performed.

[0085] Step S324, add 1 to i.

[0086] After step S324, steps S319 to S320 may be performed again.

[0087] If it is determined in step S323 that i is greater than or equal to N, step S325 may be performed.

[0088] Step S325 , determining whether the data set has been modified after the latest determination of the initial correspondence relationship.

[0089] That is, the determination of whether the data set has been modified after the latest determination of the initial correspondence may also be the determination of whether steps S321 to S322 have been performed after the most recent step S318.

[0090] If the data set has not been modified since the latest initial correspondence, Figure 4 The method 400 is shown. On the contrary, if the data set is modified after the initial correspondence is determined, step S318 is performed again.

[0091] The initial correspondence determined in step S318 can be understood as a model determined based on the detection results of each preset area. The data set can be understood as the detection result, or it can also be referred to as the detection data set. If the detection results of a preset area are inconsistent with the model and there is a deviation, if the preset area is no longer considered in the process of re-determining the model, the determined model may have a large error for the preset area.

[0092] The method provided in the embodiments of the present application determines, based on the model, a more reasonable image acquisition position for a preset region where there is a significant discrepancy between the detection results and the results determined by the model, and re-acquires the image. Thus, during the process of re-determining the model, the detection results for the preset region in the detection results are replaced with the detection results determined based on the re-acquired images, making the re-determined model more accurate.

[0093] Figure 4 This is a schematic flowchart of an image processing method provided in an embodiment of the present application. Figure 4 The method 400 shown includes steps S410 to S430, and each step is described below.

[0094] Step S410: Control the first camera to move in a first plane perpendicular to the direction of the first camera and capture an image. The first object is recorded in a first image captured by the first camera.

[0095] Step S420 : determining a first acquisition displacement corresponding to the first pixel displacement according to the correspondence between the pixel displacement and the acquisition displacement, where the first pixel displacement is the displacement of the first pixel position of the first object in the first image relative to the center of the first image.

[0096] Step S430 : determining a first position of the first object according to the first acquisition displacement and the position of the first camera when acquiring the first image.

[0097] The correspondence between the pixel displacement and the acquisition displacement used in step S420 can be obtained by Figure 1 or Figure 3 The image processing method shown is determined.

[0098] That is to say, the corresponding relationship is determined based on multiple detection pixel displacements and the detection sampling displacement corresponding to each detection pixel displacement. Among the multiple detection pixel displacements, the i-th detection pixel displacement is the displacement of the pixel position of the detection object in the i-th second detection image in the multiple second detection images relative to the center of the i-th second detection image. The i-th detection sampling displacement corresponding to the i-th detection pixel displacement is the displacement of the position of the third camera when collecting the i-th second detection image relative to the position of the third camera when collecting the first detection image. The multiple second detection images and the first detection image are all collected by the third camera during the movement of the third plane. The third plane is perpendicular to the direction of the third camera.

[0099] pass Figure 1 or Figure 3 The correspondence between the pixel displacement and the acquisition displacement determined by the method is applicable to processing an image obtained by moving a camera identical to the third camera in a first plane and acquiring an image of an object whose distance between the first plane and the object is a second distance, where the second distance is equal to the distance between the third plane and the detection object.

[0100] Therefore, if the distance between the third plane and the detection object is the first distance, and the distance between the first plane and the first object is the second distance, then the first distance is equal to the second distance. In addition, the third camera and the first camera are the same camera.

[0101] The two cameras are identical, meaning they have the same image sensor parameters and the same optical parameters. Image sensor parameters include sensor type, number of pixels, and pixel size. Camera optical parameters include focal length, field of view, and distortion parameters.

[0102] In step S430, the first position of the first object may be calculated according to the first acquisition displacement and the position of the first camera when acquiring the first image.

[0103] Alternatively, in step S430, the first camera may be controlled to move according to a first acquisition displacement and acquire a verification image. Based on the correspondence between the pixel displacement and the acquisition displacement, a verification acquisition displacement corresponding to the verification pixel displacement may be determined. The verification pixel displacement is the displacement of the verification pixel position of the first object in the verification image relative to the center of the verification image. Based on the verification acquisition displacement, the first position of the first object may be determined.

[0104] The first position of the first object determined in step S430 may be the position of the first object in a preset coordinate system, or may be the position of the first object relative to the second object.

[0105] Before step S430, a second camera that is the same as the third camera may be controlled to move in the first plane so that the second object is recorded in the second image captured by the second camera. The direction of the second camera is the same as that of the first camera.

[0106] determining, based on a correspondence between the pixel displacement and the acquisition displacement, a second acquisition displacement corresponding to a second pixel displacement, where the second pixel displacement is a displacement of a second pixel position of the second object in the second image relative to a center of the first image; Thus, in step S430, the relative position relationship between the first object and the second object is determined based on the first acquisition displacement, the second acquisition displacement, the first camera position, and the second camera position. The second acquisition position is the position of the second camera when acquiring the second image.

[0107] It should be understood that the first camera, the second camera, and the third camera are the same camera, but may or may not be the same camera.

[0108] In some embodiments, during the image processing process, different cameras may be used to capture images of the first object and the second object. Figure 1 or Figure 3 The method shown determines the correspondence between the camera's pixel displacement and acquisition displacement. After a first image is captured by a first camera, the first acquisition displacement can be determined based on the correspondence between the first camera's pixel displacement and the acquisition displacement. After a second image is captured by a second camera, the second acquisition displacement can be determined based on the correspondence between the second camera's pixel displacement and the acquisition displacement. Thus, the relative positional relationship between the first object and the second object is determined based on the first acquisition displacement, the second acquisition displacement, the first camera position, and the second camera position.

[0109] The image processing method provided in the embodiment of the present application can improve the accuracy of positioning.

[0110] Combined with the above Figures 1 to 4 , describes the image processing method of the embodiment of the present application in detail, and will be combined with Figure 5 and Figure 6 , describing the device embodiments of the present application in detail. It should be understood that the image processing device in the embodiments of the present application can execute the various methods of the aforementioned embodiments of the present application, that is, the specific working processes of the following various products can refer to the corresponding processes in the aforementioned method embodiments.

[0111] Figure 5 FIG. 5 is a schematic structural diagram of an image processing apparatus provided in an embodiment of the present application. The image processing apparatus 500 may include a control unit 510 and a processing unit 520 .

[0112] In some embodiments, the image processing apparatus 500 can be used to implement Figure 4 The steps of the method shown.

[0113] The control unit 510 is configured to control the first camera to move in a first plane perpendicular to the direction of the first camera and to capture an image, wherein a first object is recorded in a first image captured by the first camera.

[0114] The processing unit 520 is configured to determine a first acquisition displacement corresponding to a first pixel displacement according to a correspondence between pixel displacement and acquisition displacement, where the first pixel displacement is a displacement of a first pixel position of the first object in the first image relative to a center of the first image.

[0115] The processing unit 520 is configured to determine a first position of the first object according to the first acquisition displacement and a first camera position at which the first camera acquires the first image.

[0116] The corresponding relationship is determined based on a detection data set, where the detection data set includes a detection pixel displacement and a detection acquisition displacement corresponding to each of a plurality of preset areas, wherein the detection pixel displacement corresponding to any preset area represents a displacement of a position of the detection object in a second detection image in which the detection object is located in the preset area relative to a center of the second detection image, and the detection sampling displacement corresponding to any preset area represents a displacement of a position of the third camera when capturing the second detection image in which the detection object is located in the preset area relative to a first position of the third camera when capturing the first detection image. The plurality of second detection images and the first detection image in which the detection object is located in the plurality of preset areas are all captured by the third camera during movement in a third plane, the third plane being perpendicular to the orientation of the third camera, the distance between the third plane and the detection object being a first distance, the distance between the first plane and the first object being a second distance, the first distance being equal to the second distance, and the third camera being the same camera as the first camera.

[0117] In other embodiments, the image processing apparatus 500 may be used to implement Figure 1 or Figure 3 The steps of the method shown.

[0118] The control unit 510 is used to control the third camera to move in a third plane perpendicular to the direction of the third camera, and control the third camera to capture a first detection image and multiple second detection images, wherein the detection object recorded in the first detection image is located at the center of the first detection image, and the detection object in different second detection images is located in different preset areas.

[0119] The processing unit 520 is used to determine the correspondence between the pixel displacement and the acquisition displacement based on the detection data set, wherein the detection data set includes the detection pixel displacement and the detection acquisition displacement corresponding to each preset area of ​​the multiple preset areas, wherein the detection pixel displacement corresponding to any preset area represents the displacement of the position of the detection object in the second detection image in which the detection object is located in the any preset area relative to the center of the second detection image, and the detection acquisition displacement corresponding to any preset area represents the detection acquisition displacement of the position of the third camera when capturing the second detection image in which the detection object is located in the any preset area relative to the first position of the third camera when capturing the first detection image.

[0120] The corresponding relationship is used to determine a first acquisition displacement corresponding to a first pixel displacement, where the first pixel displacement is the displacement of a first pixel position of a first object in the first image captured by the first camera relative to the center of the first image, and the first image is captured by the first camera during movement in a first plane perpendicular to the direction of the first camera. The first acquisition displacement and the position of the first camera when capturing the first image are used to determine the first position of the first object.

[0121] It should be noted that the image processing device 500 is implemented in the form of a functional unit. The term "unit" here can be implemented in the form of software and / or hardware, and is not specifically limited to this.

[0122] For example, a "unit" may be a software program, a hardware circuit, or a combination of the two that implements the aforementioned functionality. The hardware circuit may include an application specific integrated circuit (ASIC), an electronic circuit, a processor (e.g., a shared processor, a dedicated processor, or a group of processors) and memory for executing one or more software or firmware programs, combined logic circuits, and / or other suitable components that support the described functionality.

[0123] Therefore, the units of each example described in the embodiments of this application can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0124] Figure 6 Schematic diagram of the structure of an electronic device 600 provided in an embodiment of the present application. Figure 6 As shown, the electronic device 600 of this embodiment includes: at least one processor 601 ( Figure 6 Only one processor is shown in the figure), a memory 602, and a computer program 603 stored in the memory 602 and executable on the at least one processor 601, wherein the processor 601 implements the steps of any of the above method embodiments when executing the computer program 603.

[0125] Those skilled in the art will understand that Figure 6 This is merely an example of the electronic device 600 and does not constitute a limitation on the electronic device 600 . The electronic device 600 may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the electronic device 600 may also include input and output devices, network access devices, etc.

[0126] The processor 601 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. A general-purpose processor may be a microprocessor or any conventional processor.

[0127] In some embodiments, the memory 602 may be an internal storage unit of the electronic device 600, such as a hard drive or memory of the electronic device 600. In other embodiments, the memory 602 may also be an external storage device of the electronic device 600, such as a plug-in hard drive, a Smart Media Card (SMC), a Secure Digital (SD) card, a flash memory card, etc. equipped on the electronic device 600. Furthermore, the memory 602 may include both an internal storage unit of the electronic device 600 and an external storage device. The memory 602 is used to store an operating system, application programs, a boot loader, data, and other programs, such as the program code of the computer program. The memory 602 may also be used to temporarily store data that has been output or is about to be output.

[0128] It should be noted that the information interaction, execution process, etc. between the above-mentioned devices / units are based on the same concept as the method embodiment of this application. Their specific functions and technical effects can be found in the method embodiment section and will not be repeated here.

[0129] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0130] An embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, any of the above method embodiments can be implemented.

[0131] An embodiment of the present application provides a computer program product, which can implement any of the above method embodiments when the computer program product is running.

[0132] If the integrated unit is implemented as a software functional unit and sold or used as a standalone product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application can implement all or part of the process steps in the above-mentioned method embodiments by using a computer program to instruct the relevant hardware. The computer program can be stored in a computer-readable storage medium. When executed by a processor, the computer program can implement the steps of each of the above-mentioned method embodiments. The computer program includes computer program code, which can be in source code form, object code form, executable file, or some intermediate form. The computer-readable medium can include at least: any entity or device capable of carrying the computer program code to the camera / terminal device, recording medium, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media. Examples include USB flash drives, removable hard drives, magnetic disks, or optical disks. In some possible implementations, the computer-readable medium cannot be an electrical carrier signal or telecommunication signal.

[0133] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant description of other embodiments.

[0134] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0135] In the embodiments provided in this application, it should be understood that the disclosed systems and methods can be implemented in other ways. For example, the system embodiments described above are merely schematic. For example, the division of the modules or units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0136] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0137] The above-described embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present application, and should all be included in the scope of protection of the present application.

Claims

1. An image processing method, characterized in that: The method comprises: Controlling a first camera to move in a first plane perpendicular to a direction of the first camera and to capture an image, wherein a first object is recorded in a first image captured by the first camera; determining, according to a correspondence between pixel displacement and acquisition displacement, a first acquisition displacement corresponding to a first pixel displacement, where the first pixel displacement is a displacement of a first pixel position of the first object in the first image relative to a center of the first image; determining a first position of the first object according to the first acquisition displacement and a first camera position at which the first camera acquires the first image; The corresponding relationship is determined based on a detection data set, wherein the detection data set includes a detection pixel displacement and a detection acquisition displacement corresponding to each of a plurality of preset areas. The detection pixel displacement corresponding to any preset area represents the displacement of the position of the detection object in a second detection image in which the detection object is located in the preset area relative to the center of the second detection image. The detection sampling displacement corresponding to any preset area represents the displacement of the position of the third camera when capturing the second detection image in which the detection object is located in the preset area relative to the first position of the third camera when capturing the first detection image. The plurality of second detection images and the first detection image in which the detection object is located in the plurality of preset areas are all captured by the third camera during movement in a third plane, the third plane is perpendicular to the direction of the third camera, the distance between the third plane and the detection object is a first distance, the distance between the first plane and the first object is a second distance, the first distance is equal to the second distance, and the third camera is the same camera as the first camera.

2. The method according to claim 1, characterized in that The corresponding relationship is determined by a first operation, which includes: determining an initial correspondence between pixel displacement and acquisition displacement based on the detection data set; Determining, based on the initial correspondence, a predicted sampling displacement corresponding to each detection pixel displacement in the detection data set, wherein the predicted sampling displacement corresponding to any detection pixel displacement and the any detection pixel displacement correspond to the same preset area; When the difference between the detected sampling displacement and the predicted sampling displacement corresponding to each preset area in the detection data set is less than a difference threshold, determining the initial corresponding relationship as the corresponding relationship; If at least one target area exists in the multiple preset areas, a second operation is performed, and a difference between a detected sampling displacement and a predicted sampling displacement corresponding to any one of the at least one target area is greater than or equal to a difference threshold. The second operation includes: For any target area, controlling the third camera to move in the third plane to a position whose displacement relative to the first position is the predicted sampling displacement corresponding to the any target area, and capturing a third detection image corresponding to the any target area; Updating the detection acquisition displacement corresponding to any target area in the detection data set to the predicted sampling displacement corresponding to the any target area, and updating the detection pixel displacement corresponding to the any target area to the displacement of the position of the detection object in the third detection image corresponding to the any target area relative to the center of the third detection image; After the detection data set is updated based on each target area, the corresponding relationship is determined according to the updated detection data set.

3. The method according to claim 2, characterized in that After updating the detection data set based on each target area, determining the corresponding relationship according to the updated detection data set includes: After the detection data set is updated based on each target area, the initial correspondence is re-determined based on the updated detection data set, and the operation of determining the predicted sampling displacement corresponding to each detection pixel displacement in the updated detection data set is re-performed until the difference between the detection sampling displacement and the predicted sampling displacement corresponding to each preset area in the updated detection data set is less than the difference threshold.

4. The method according to any one of claims 1 to 3, characterized in that The plurality of preset areas include other areas except the area where the center of the image is located, among the plurality of areas obtained by evenly dividing the image captured by the third camera.

5. The method according to any one of claims 1 to 3, characterized in that The corresponding relationship includes a first proportional relationship between the pixel displacement and the acquisition displacement along a first direction, and a second proportional relationship between the pixel displacement and the acquisition displacement along a second direction, wherein the first direction is perpendicular to the second direction.

6. The method according to any one of claims 1 to 3, characterized in that Determining the first position of the first object according to the first acquired displacement includes: Controlling the first camera to move according to the first acquisition displacement and acquire a first verification image; Determining, based on the corresponding relationship, a verification acquisition displacement corresponding to a verification pixel displacement, wherein the verification pixel displacement is a displacement of a verification pixel position of the first object in the verification image relative to a center of the verification image; The first position is determined according to the verification acquisition displacement.

7. The method according to any one of claims 1 to 3, characterized in that The method further comprises: Controlling a second camera to move in the first plane so that a second object is recorded in a second image captured by the second camera, the second camera is oriented in the same direction as the first camera, and a third distance between the second object and the first plane is equal to the first distance; determining, based on a correspondence between the pixel displacement and the acquisition displacement, a second acquisition displacement corresponding to a second pixel displacement, where the second pixel displacement is a displacement of a second pixel position of the second object in the second image relative to a center of the first image; The determining the first position of the first object according to the first acquisition displacement and a first camera position at which the first camera acquires the first image includes: The relative position relationship between the first object and the second object is determined according to the first acquisition displacement, the second acquisition displacement, the first camera position and the second camera position, where the second camera position is the position of the second camera when acquiring the second image.

8. An image processing method, characterized in that: The method comprises: Controlling a third camera to move in a third plane perpendicular to the direction of the third camera, and controlling the third camera to capture a first detection image and a plurality of second detection images, wherein the detection object recorded in the first detection image is located at the center of the first detection image, and the detection object in different second detection images is located in different preset areas; Determining a correspondence between a pixel displacement and an acquisition displacement based on a detection data set, the detection data set including a detection pixel displacement and a detection acquisition displacement corresponding to each of a plurality of preset areas, wherein the detection pixel displacement corresponding to any one of the preset areas represents a displacement of a position of the detection object in a second detection image in which the detection object is located in the preset area relative to a center of the second detection image, and the detection acquisition displacement corresponding to any one of the preset areas represents a detection acquisition displacement of a position of the third camera when capturing the second detection image in which the detection object is located in the preset area relative to a first position of the third camera when capturing the first detection image; The corresponding relationship is used to determine a first acquisition displacement corresponding to a first pixel displacement, where the first pixel displacement is the displacement of a first pixel position of a first object in a first image captured by a first camera relative to the center of the first image, the first image being captured by the first camera while moving in a first plane perpendicular to the direction of the first camera, the first acquisition displacement and the position of the first camera when capturing the first image are used to determine a first position of the first object, the distance between the third plane and the detection object is a first distance, the distance between the first plane and the first object is a second distance, the first distance is equal to the second distance, and the third camera is the same camera as the first camera.

9. The method according to claim 8, characterized in that Determining the correspondence between pixel displacement and acquisition displacement based on the detection data set includes: determining an initial correspondence between pixel displacement and acquisition displacement based on the detection data set; Determining, based on the initial correspondence, a predicted sampling displacement corresponding to each detection pixel displacement in the detection data set, wherein the predicted sampling displacement corresponding to any detection pixel displacement and the any detection pixel displacement correspond to the same preset area; When the difference between the detected sampling displacement and the predicted sampling displacement corresponding to each preset area in the detection data set is less than a difference threshold, determining the initial corresponding relationship as the corresponding relationship; If at least one target area exists in the multiple preset areas, a second operation is performed, and a difference between a detected sampling displacement and a predicted sampling displacement corresponding to any one of the at least one target area is greater than or equal to a difference threshold. The second operation includes: For any target area, controlling the third camera to move in the third plane to a position whose displacement relative to the first position is the predicted sampling displacement corresponding to the any target area, and capturing a third detection image corresponding to the any target area; Updating the detection acquisition displacement corresponding to any target area in the detection data set to the predicted sampling displacement corresponding to the any target area, and updating the detection pixel displacement corresponding to the any target area to the displacement of the position of the detection object in the third detection image corresponding to the any target area relative to the center of the third detection image; After the detection data set is updated based on each target area, the corresponding relationship is determined according to the updated detection data set.

10. An electronic device, characterized in that: The electronic device comprises a processor and a memory, wherein the memory is used to store a computer program, and the processor is used to call and run the computer program from the memory, so that the electronic device executes the method according to any one of claims 1 to 9.

Images