Image processing methods and related equipment

Abstract

This application provides an image processing method and related apparatus, relating to the field of image processing. The method includes: controlling a first camera to move on a first plane perpendicular to its orientation and to acquire an image, wherein a first object is recorded in the first image acquired by the first camera; determining a first acquisition displacement corresponding to a first pixel displacement based on a correspondence between pixel displacement and acquisition displacement, wherein the first pixel displacement is the displacement of the first pixel position of the first object in the first image relative to the center of the first image; and determining a first position of the first object based on the first acquisition displacement and the position of the first camera when acquiring the first image, wherein the correspondence is determined based on multiple detection images of the detected object acquired by the camera, and the position of the camera when acquiring each detection image. This method can improve the accuracy of the determined object position.

Description

Technical Field

[0001] This application relates to the field of image processing, and more particularly to an image processing method and related equipment. Background Technology

[0002] By acquiring images of objects, the position of an object can be detected without physical contact. This non-contact position detection method eliminates the need for personnel or probes to enter hazardous areas during the measurement process, thus providing an irreplaceable advantage in fields such as precision manufacturing, life sciences, autonomous driving, hazardous environments, and large-scale logistics.

[0003] Improving the accuracy of object location results is a problem that urgently needs to be solved. Summary of the Invention

[0004] This application provides an image processing method and related equipment that can improve the accuracy of positioning results.

[0005] In a first aspect, an image processing method is provided, comprising: controlling a first camera to move on a first plane perpendicular to the orientation of the first camera and to acquire an image, wherein a first object is recorded in the first image acquired by the first camera; determining a first acquisition displacement corresponding to a first pixel displacement based on a correspondence between pixel displacement and acquisition displacement, wherein the first pixel displacement is the displacement of a first pixel position of the first object in the first image relative to the center of the first image; determining a first position of the first object based on the first acquisition displacement and the position of the first camera when acquiring the first image; wherein the correspondence is determined based on a plurality of detected pixel displacements and a detection sampling displacement corresponding to each detected pixel displacement, wherein the correspondence is determined based on a detection data set, the detection data set including a plurality of preset regions, each preset region corresponding to a detected pixel displacement and a detection acquisition displacement, wherein... In this context, the detection pixel displacement corresponding to any preset region represents the displacement of the detected object's position relative to the center of the second detection image in which the detected object is located within the preset region. The detection sampling displacement corresponding to any preset region represents the displacement of the third camera's position relative to the first position of the third camera when acquiring the second detection image in which the detected object is located within the preset region, while acquiring the first detection image. The multiple second detection images and the first detection image in which the detected object is located in the multiple preset regions are all acquired by the third camera during its movement along a third plane. The third plane is perpendicular to the orientation of the third camera. The distance between the third plane and the detected object is a first distance, and the distance between the first plane and the first object is a second distance. The first distance and the second distance are equal. The third camera and the first camera are the same camera.

[0006] In some possible implementations, the correspondence is determined through a first operation, which includes: determining an initial correspondence between pixel displacements and acquisition displacements based on the detection data set; determining a predicted sampling displacement corresponding to each detected pixel displacement in the detection data set based on the initial correspondence, wherein the predicted sampling displacement corresponding to any detected pixel displacement corresponds to the same preset region; determining the initial correspondence as the correspondence if the difference between the detected sampling displacement and the predicted sampling displacement corresponding to each preset region in the detection data set is less than a difference threshold; and performing a second operation if at least one target region exists among the multiple preset regions, wherein the detected sampling displacement corresponding to any one of the at least one target region is determined as the corresponding region. If the difference between the displacement and the predicted sampling displacement is greater than or equal to a difference threshold, the second operation includes: for any target region, controlling the third camera to move on the third plane to a position relative to the first position where the displacement is the predicted sampling displacement corresponding to the target region, and acquiring a third detection image corresponding to the target region; updating the detection acquisition displacement corresponding to the target region in the detection data set to the predicted sampling displacement corresponding to the target region, and updating the detection pixel displacement corresponding to the target region to the displacement of the position of the detected object in the third detection image corresponding to the target region relative to the center of the third detection image; after updating the detection data set based on each target region, determining the correspondence based on the updated detection data set.

[0007] In some possible implementations, the step of determining the correspondence based on the updated detection data set after updating the detection data set based on each target region includes: after updating the detection data set based on each target region, redetermining the initial correspondence based on the updated detection data set, and re-performing the predicted sampling displacement corresponding to the displacement of each detected pixel until the difference between the detection sampling displacement and the predicted sampling displacement corresponding to each second detection image in the detection data set is less than the difference threshold.

[0008] In some possible implementations, the plurality of preset regions include regions other than the region where the center of the image is located, which are obtained by uniformly dividing the image captured by the third camera.

[0009] In some possible implementations, the correspondence includes a first proportional relationship between the pixel displacement and the acquisition displacement along a first direction, and a second proportional relationship between the pixel displacement and the acquisition displacement along a second direction, wherein the first direction is perpendicular to the second direction.

[0010] In some possible implementations, determining the first position of the first object based on the first acquisition displacement includes: controlling the first camera to move according to the first acquisition displacement and acquiring a first verification image; determining the verification acquisition displacement corresponding to the verification pixel displacement according to the correspondence, wherein the verification pixel displacement is the displacement of the verification pixel position of the first object in the verification image relative to the center of the verification image; and determining the first position based on the verification acquisition displacement.

[0011] In some possible implementations, the method further includes: controlling the second camera to move on the first plane such that a second object is recorded in the second image captured by the second camera, the orientation of the second camera being the same as that of the first camera, and a third distance between the second object and the first plane being equal to the first distance; determining a second acquisition displacement corresponding to a second pixel displacement based on the correspondence between pixel displacement and acquisition displacement, wherein the second pixel displacement is the displacement of the second pixel position of the second object in the second image relative to the center of the first image; and determining the first position of the first object based on the first acquisition displacement and the first camera position when the first camera captures the first image, including: determining the relative positional relationship between the first object and the second object based on the first acquisition displacement, the second acquisition displacement, the first camera position, and the second camera position, wherein the second camera position is the position of the second camera when capturing the second image.

[0012] Secondly, embodiments of this application provide an image processing method, comprising: controlling a third camera to move in a third plane perpendicular to the orientation of the third camera, and controlling the third camera to acquire a first detection image and a plurality of second detection images, wherein a detected object recorded in the first detection image is located at the center of the first detection image, and the detected object in different second detection images is located in different preset regions; determining the correspondence between pixel displacement and acquisition displacement according to a detection data set, wherein the detection data set includes a detection pixel displacement and a detection acquisition displacement corresponding to each of the plurality of preset regions, wherein the detection pixel displacement corresponding to any preset region represents the displacement of the detected object in the second detection image located in the second detection image relative to the center of the second detection image, and the detection acquisition displacement corresponding to any preset region represents the acquisition of the detected object located in the second detection image located in the preset region. The detection acquisition displacement of the third camera relative to the first position of the third camera when acquiring the first detection image in any preset area's second detection image; the correspondence is used to determine the first acquisition displacement corresponding to the first pixel displacement, the first pixel displacement being the displacement of the first pixel position of the first object in the first image acquired by the first camera relative to the center of the first image, the first image being acquired by the first camera while moving on a first plane perpendicular to the orientation of the first camera, the first acquisition displacement and the position of the first camera when acquiring the first image being used to determine the first position of the first object, the distance between the third plane and the detection object being the first distance, the distance between the first plane and the first object being the second distance, the first distance being equal to the second distance, and the third camera being the same camera as the first camera.

[0013] In some possible implementations, determining the correspondence between pixel displacements and acquisition displacements based on the detection data set includes: determining an initial correspondence between pixel displacements and acquisition displacements based on the detection data set; determining a predicted sampling displacement corresponding to each detected pixel displacement in the detection data set based on the initial correspondence, wherein the predicted sampling displacement corresponding to any detected pixel displacement corresponds to the same preset region; determining the initial correspondence as the correspondence when the difference between the detected sampling displacement and the predicted sampling displacement corresponding to each preset region in the detection data set is less than a difference threshold; and performing a second operation when at least one target region exists among the multiple preset regions, wherein the detection of any target region among the at least one target region... If the difference between the sampled displacement and the predicted sampled displacement is greater than or equal to a difference threshold, the second operation includes: for any target region, controlling the third camera to move on the third plane to a position relative to the first position that is the predicted sampled displacement corresponding to the target region, and acquiring a third detection image corresponding to the target region; updating the detection acquisition displacement corresponding to the target region in the detection data set to the predicted sampled displacement corresponding to the target region, and updating the detection pixel displacement corresponding to the target region to the displacement of the position of the detected object in the third detection image corresponding to the target region relative to the center of the third detection image; after updating the detection data set based on each target region, determining the correspondence based on the updated detection data set.

[0014] Thirdly, embodiments of this application provide an image processing apparatus, including a unit for performing the method of the first aspect or the second aspect.

[0015] Fourthly, embodiments of this application provide an electronic device, including a processor and a memory, wherein the memory is used to store a computer program, and the processor is used to call and run the computer program from the memory, causing the electronic device to perform the method of the first aspect or the second aspect.

[0016] Fifthly, embodiments of this application provide a computer-readable storage medium storing a computer program, which, when executed, causes the methods described in the first or second aspect to be performed.

[0017] Sixthly, embodiments of this application provide a computer program product, the computer program product including computer program instructions, which, when executed, cause the methods of the first or second aspect described above to be performed.

[0018] In a seventh aspect, embodiments of this application provide a chip including a processor and a data interface, wherein the processor reads instructions stored in a memory through the data interface to implement the methods of the first or second aspect described above.

[0019] Eighthly, embodiments of this application provide an image processing apparatus, including a first sensor and a processor. The first sensor is used to acquire a first image and first pose information; the processor is used to execute the image processing method described in the first aspect.

[0020] The beneficial effects of this application embodiment compared with the prior art are as follows: by acquiring images of the detected object on a plane perpendicular to the camera orientation using a camera, and based on the displacement of the camera position relative to the camera position at the center of the image, which makes the detected object located in the image, and the displacement of the detected object position relative to the center of the image, the correspondence between pixel displacement and acquisition displacement is determined. Based on this correspondence, the first acquisition displacement corresponding to the first pixel displacement of the first object position relative to the center of the image is determined. Based on the first acquisition displacement and the position of the first object when the camera acquires the image, the position of the first object is determined, thereby improving the accuracy of the determined object position. Attached Figure Description

[0021] Figure 1 This is a schematic flowchart of an image processing method provided in an embodiment of this application;

[0022] Figure 2 This is a schematic diagram of the camera performing image acquisition according to an embodiment of this application;

[0023] Figure 3 This is a schematic flowchart of an image processing method provided in an embodiment of this application;

[0024] Figure 4 This is a schematic flowchart of an image processing method provided in an embodiment of this application;

[0025] Figure 5 This is a schematic structural diagram of an image processing apparatus provided in an embodiment of this application;

[0026] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0027] The technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings.

[0028] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0029] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0030] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0031] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0032] References to "one embodiment" or "some embodiments" in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized.

[0033] By acquiring images, the relative positions of multiple objects can be detected without physical contact. This non-contact phase position measurement method eliminates the need for personnel or probes to enter hazardous areas during the measurement process, thus providing an irreplaceable advantage in fields such as precision manufacturing, life sciences, autonomous driving, hazardous environments, and large-scale logistics.

[0034] Improving the accuracy of measurement results is an urgent problem to be solved.

[0035] In view of this, this application provides a related image processing scheme that can improve the accuracy of detection results. The scheme provided in this application is described below.

[0036] Figure 1 This is a schematic flowchart of an image processing method provided in an embodiment of this application. Figure 1 The method shown includes steps S101 to S105.

[0037] Step S101: Control the third camera to move on a third plane perpendicular to the orientation of the third camera, and control the third camera to acquire a first detection image and multiple second detection images, wherein the detection object recorded in the first detection image is located at the center of the first detection image, and the detection object in different second detection images is located in different preset areas.

[0038] The first detection image and the multiple second detection images are all captured by a third camera. The first detection image and the multiple second detection images are of the same size.

[0039] As the third camera moves along the third plane, the relative position of the third camera and the object being detected changes, so that the location of the object being detected can be different in the image captured by the third camera at different times.

[0040] The number of secondary detection images can be preset. For example... Figure 2 As shown, in different second detection images captured by the third camera, the detected object is located in different preset regions. There may be no overlap between adjacent preset regions. These multiple preset regions can cover the entire image captured by the third camera.

[0041] The image captured by the third camera can be rectangular. The image captured by the third camera is divided into multiple regions. These multiple preset regions can be the regions obtained from this division. Alternatively, the multiple preset regions can be multiple regions other than the region at the center of the image. The sizes of these multiple preset regions can be equal or unequal.

[0042] In other words, the multiple preset regions include areas other than the center of the image, which are evenly divided into multiple regions based on the image captured by the third camera. The multiple preset regions may also include the center of the image.

[0043] For example, such as Figure 2 As shown, the image captured by the third camera can be evenly divided into nine regions of equal size, arranged in a 9-grid pattern (three rows and three columns). These nine regions can each be a uniform rectangle. Each rectangle can serve as a preset region. Alternatively, the eight rectangular regions excluding the region containing the center point of the image can each serve as a preset region.

[0044] Step S102: Determine the correspondence between pixel displacement and acquisition displacement based on the detection data set. The detection data set includes the detection pixel displacement and detection acquisition displacement corresponding to each of the plurality of preset regions. The detection pixel displacement corresponding to any preset region represents the displacement of the position of the detected object in the second detection image of the second detection image relative to the center of the second detection image. The detection acquisition displacement corresponding to any preset region represents the detection acquisition displacement of the position of the third camera when acquiring the second detection image of the detected object located in the second detection image of the second detection image relative to the first position of the third camera when acquiring the first detection image.

[0045] Based on the detection data set, the initial correspondence between pixel displacement and acquisition displacement can be determined. This initial correspondence can serve as the basis for determining the relationship between pixel displacement and acquisition displacement. Figure 4 The image processing method shown.

[0046] Alternatively, after determining the initial correspondence, the predicted sampling displacement corresponding to each detected pixel displacement in the detection data set can be determined based on the initial correspondence. The predicted sampling displacement corresponding to any detected pixel displacement corresponds to the same preset region.

[0047] When the difference between the detected sampling displacement and the predicted sampling displacement corresponding to each preset region in the detection dataset is less than the difference threshold, the initial correspondence is determined as follows: Figure 4 The correspondences used in the image processing methods shown.

[0048] If the difference between the detection sampling displacement and the predicted sampling displacement corresponding to each preset area in the detection dataset is not all less than the difference threshold, the image can be acquired again by the third camera based on the predicted sampling displacement, and the initial correspondence can be re-determined.

[0049] In a detection dataset, if the difference between the detected sampling displacement and the predicted sampling displacement corresponding to each preset region is not all less than a difference threshold, multiple preset regions may include at least one target region. The difference between the detected sampling displacement and the predicted sampling displacement corresponding to any target region is greater than or equal to the difference threshold.

[0050] For any target area, the displacement of the third camera on the third plane relative to the first position can be controlled to be the position of the predicted sampling displacement corresponding to the target area, and the third detection image corresponding to the target area can be acquired.

[0051] Then, the detection acquisition displacement corresponding to any target region in the detection data set is updated to the predicted sampling displacement corresponding to any target region, and the detection pixel displacement corresponding to any target region is updated to the displacement of the position of the detected object in the third detection image corresponding to any target region relative to the center of the third detection image.

[0052] After updating the detection data set based on each target region, the initial correspondence can be redefined based on the updated detection data set.

[0053] It should be understood that by reasonably controlling the position of the third camera, so that the distance between the detected object and the edge of the preset area in each second detection image is greater than a preset value, the predicted sampling displacement corresponding to any preset area determined based on the initial correspondence can be located in that preset area.

[0054] For example, if the predicted sampling displacement of each target region is located within that target region, a third detection image can be acquired for each target region, and the initial correspondence can be re-determined based on the predicted third detection image. However, if the predicted sampling displacement of a target region is located outside that target region, a second detection image can be acquired for each preset region again, and the initial correspondence can be determined based on the re-acquired second detection image.

[0055] The initial image capture position, i.e., the position where the third camera captures the second detection image, can be manually set or randomly determined. That is, the values ​​are scattered and lack a systematic correlation. The predicted sampling displacement corresponding to each region is an ideal value calculated based on the sampling results of each preset region. A systematic relationship is established between the predicted sampling displacements corresponding to each region. Therefore, the third detection image is captured at the position corresponding to the predicted sampling displacement of the target region relative to the first position, making the capture position of the third detection image more reasonable. Consequently, the relationship between the detection capture displacement and the detection pixel displacement of the third detection image is more rational.

[0056] After redetermining the initial correspondence based on the updated detection data set, the redetermined initial correspondence can be used as the correspondence for... Figure 4 The image processing method shown is as follows. Alternatively, after determining the initial correspondence each time, based on the initial correspondence, the predicted sampling displacement corresponding to the displacement of each detected pixel in the detection dataset is determined until the difference between the detected sampling displacement and the predicted sampling displacement corresponding to each preset region in the detection dataset is less than the difference threshold.

[0057] Below, in conjunction with Figure 3 , Figure 1 The image processing method shown will be described in detail.

[0058] Figure 3 This is a schematic flowchart of an image processing method provided in an embodiment of this application. Figure 3 The method shown includes steps S311 to S325.

[0059] Step S311: Control the third camera to move to position pc0 directly above the detection object on the first plane and acquire the first detection image.

[0060] Therefore, the object to be detected in the first detection image is located at the center Pp0 of the first detection image.

[0061] Before step S311, i is 1. That is, the initial value of i is 1.

[0062] Step S312: Control the third camera to move on the first plane and acquire images to obtain the i-th second detection image, in which the detected object is located in the i-th preset area.

[0063] As the third camera moves along the first plane, its orientation remains perpendicular to the first plane.

[0064] The third camera can detect whether an object is located in the i-th preset area in the image it captures.

[0065] If the detected object is not located in the i-th preset area, the third camera can be controlled to move on the first plane and capture images again until the detected object is located in the i-th preset area in the image captured by the third camera.

[0066] Step S313: Based on the actual acquisition position Pci of the third sensor when acquiring the i-th second detection image, determine the acquisition displacement ΔPci of the actual acquisition position Pci of the third sensor when acquiring the i-th second detection image relative to the position Pc0 when acquiring the first detection image.

[0067] The actual acquisition position Pci of the third sensor in the i-th second detection image can be represented as the coordinates in a Cartesian coordinate system located in the third plane. For example, taking the position Pc0 of the third sensor when acquiring the first detection image as the origin, the actual acquisition position Pci of the third sensor in the i-th second detection image can be represented as the acquisition displacement ΔPci of the actual acquisition position Pci relative to the position Pc0.

[0068] Step S314: Based on the pixel position Ppi of the detected object in the i-th second detection image, determine the pixel displacement ΔPpi of the pixel position Ppi of the detected object in the second detection image relative to the center Pp0 of the second detection image.

[0069] The pixel position Ppi of the detected object in the i-th second detection image can be represented as the coordinates in the Cartesian coordinate system of the second detection image. For example, with the center Pp0 of the second detection image as the origin, the pixel position Ppi of the detected object in the i-th second detection image can be represented as the pixel displacement ΔPpi of the pixel position Ppi relative to the position Pp0.

[0070] Step S315: Add the pixel displacement ΔPpi and the acquisition displacement ΔPci to the dataset.

[0071] Step S316: Determine whether i is greater than or equal to the number N of preset regions.

[0072] If i is less than the number of preset regions, proceed to step S319. If i is greater than or equal to the number of preset regions, proceed to step S318.

[0073] Step S317: Increment i by 1.

[0074] That is, in step S317, i is set to i+1.

[0075] After step S317, step S312 is performed again.

[0076] Step S318: Determine the initial correspondence between pixel displacement and acquisition displacement based on the data set.

[0077] In other words, based on the N acquisition displacements ΔPci and the pixel displacement ΔPpi corresponding to each acquisition displacement ΔPci, the initial correspondence between the pixel displacement and the acquisition displacement can be determined.

[0078] The initial correspondence between pixel displacement and acquisition displacement can be expressed as a first proportional relationship between pixel displacement and acquisition displacement in a first direction and a second proportional relationship between pixel displacement and acquisition displacement in a second direction. The first and second directions can be mutually perpendicular. The proportional coefficients in the first and second proportional relationships can be the same or different.

[0079] When the scaling factors in the first and second proportional relationships are different, the initial correspondence can be represented by a coefficient matrix. The coefficient matrix includes the scaling factors in the first and second proportional relationships.

[0080] For example, both the acquisition displacement ΔPci and the pixel displacement ΔPpi can be represented by coordinates. The horizontal and vertical axes of the coordinate system containing the acquisition displacement ΔPci and the coordinate system containing the pixel displacement ΔPpi can be parallel. The first direction and the second direction can be the horizontal axis and the vertical axis of the coordinate system, respectively.

[0081] After step S318, i can be set to 1, and then step S319 can be performed.

[0082] Step S319: Based on the initial correspondence between pixel displacement and acquisition displacement, determine the predicted acquisition displacement ΔPci' corresponding to the pixel position Ppi of the detected object in the i-th second detection image.

[0083] Step S320: Determine whether the difference between the predicted acquisition displacement ΔPci' and the acquisition displacement ΔPci is greater than or equal to the difference threshold.

[0084] If the difference between the predicted acquisition displacement ΔPci' and the acquisition displacement ΔPci is less than the difference threshold, step S323 can be performed.

[0085] If, in step S320, it is determined that the difference between the predicted acquisition displacement ΔPci' and the acquisition displacement ΔPci is greater than or equal to the difference threshold, steps S321 to S323 can be performed.

[0086] Step S321: Delete the acquisition displacement ΔPci and the corresponding pixel displacement ΔPpi from the data set.

[0087] Step S322: Control the third camera to move on the third plane to the position Pc0 relative to the position when the third camera acquires the first detection image. The position is the predicted acquisition displacement ΔPci'. Reacquire the i-th second detection image and record the predicted acquisition displacement ΔPci' as the acquisition displacement ΔPci.

[0088] Step S323: Determine whether i is greater than or equal to the number N of preset regions.

[0089] If i is less than N, step S324 can be performed.

[0090] Step S324: Increment i by 1.

[0091] After step S324, steps S319 to S320 can be performed again.

[0092] If it is determined in step S323 that i is greater than or equal to N, step S325 can be performed.

[0093] Step S325: Determine whether the data set has been modified after the initial correspondence was determined.

[0094] In other words, determining whether the data set has been modified after the latest determination of the initial correspondence can also be done by determining whether steps S321 to S322 have been performed after the most recent step S318.

[0095] If the dataset has not been modified after the initial correspondence was determined, then... Figure 4 Method 400 is shown. Conversely, if the data set is modified after the initial correspondence has been determined, step S318 is performed again.

[0096] The initial correspondence determined in step S318 can be understood as a model determined based on the detection results of each preset region. The dataset can be understood as the detection results, or it can be called the detection dataset. If the detection results for a certain preset region are inconsistent with the model and there is a deviation, and if the preset region is no longer considered during the process of redetermining the model, it may lead to a large error in the determined model for that preset region.

[0097] The method provided in this application, based on the model, identifies a preset region where there is a significant difference between the detection result and the result determined by the model. A more reasonable image acquisition location is then determined based on the model, and image acquisition is performed again. Therefore, during the process of re-determining the model, the detection result for this preset region in the detection result is replaced with the detection result determined based on the re-acquired image, making the re-determined model more accurate.

[0098] Figure 4 This is a schematic flowchart of an image processing method provided in an embodiment of this application. Figure 4 The method 400 shown includes steps S410 to S430, which are described below.

[0099] Step S410: Control the first camera to move on a first plane perpendicular to the orientation of the first camera and perform image acquisition. The first image acquired by the first camera records the first object.

[0100] Step S420: Based on the correspondence between pixel displacement and acquisition displacement, determine the first acquisition displacement corresponding to the first pixel displacement. The first pixel displacement is the displacement of the first pixel position of the first object in the first image relative to the center of the first image.

[0101] Step S430: Determine the first position of the first object based on the first acquisition displacement and the position where the first camera was when acquiring the first image.

[0102] The correspondence between pixel displacement and acquisition displacement used in step S420 can be obtained through... Figure 1 or Figure 3 The image processing method shown is determined.

[0103] In other words, the correspondence is determined based on multiple detection pixel displacements and the corresponding detection sampling displacement for each detection pixel displacement. Specifically, the i-th detection pixel displacement is the displacement of the pixel position of the detected object in the i-th second detection image relative to the center of the i-th second detection image. The i-th detection sampling displacement corresponding to the i-th detection pixel displacement is the displacement of the position of the third camera when acquiring the i-th second detection image relative to the position of the third camera when acquiring the first detection image. Both the multiple second detection images and the first detection image are acquired by the third camera during its movement along the third plane. The third plane is perpendicular to the orientation of the third camera.

[0104] pass Figure 1 or Figure 3 The method described above determines the correspondence between pixel displacement and acquisition displacement. This method is applicable to processing images obtained by moving a camera (identical to the third camera) on a first plane and acquiring images of an object at a distance of a second distance between the first plane and the object being detected. The second distance is equal to the distance between the third plane and the detected object.

[0105] Therefore, if the distance between the third plane and the detected object is the first distance, and the distance between the first plane and the first object is the second distance, then the first distance and the second distance are equal. Furthermore, the third camera and the first camera are the same camera.

[0106] The two cameras are identical, meaning that the image sensor parameters and optical parameters of the two cameras are the same. Image sensor parameters include sensor type, number of pixels, and pixel size. Camera optical parameters include focal length, field of view, and distortion parameters.

[0107] In step S430, the first position of the first object can be calculated based on the first acquisition displacement and the position where the first camera was when acquiring the first image.

[0108] Alternatively, in step S430, the first camera can be controlled to move according to the first acquisition displacement and acquire a verification image. Based on the correspondence between pixel displacement and acquisition displacement, the verification acquisition displacement corresponding to the verification pixel displacement can be determined. The verification pixel displacement is the displacement of the verification pixel position of the first object in the verification image relative to the center of the verification image. Based on the verification acquisition displacement, the first position of the first object can be determined.

[0109] The first position of the first object determined in step S430 is the position of the first object in a certain preset coordinate system, or it can be the position of the first object relative to the second object.

[0110] Before step S430, a second camera, identical to the third camera, can be controlled to move on the first plane so that the second image captured by the second camera records the second object. The orientation of the second camera is the same as that of the first camera.

[0111] Based on the correspondence between pixel displacement and acquisition displacement, the second acquisition displacement corresponding to the second pixel displacement is determined. The second pixel displacement is the displacement of the second pixel position of the second object in the second image relative to the center of the first image.

[0112] Therefore, in step S430, the relative positional relationship between the first object and the second object is determined based on the first acquisition displacement, the second acquisition displacement, the position of the first camera, and the position of the second camera. The second acquisition position is the position where the second camera is located when acquiring the second image.

[0113] It should be understood that the first camera, the second camera, and the third camera are the same camera, but they may or may not be the same camera.

[0114] In some embodiments, during image processing, different cameras can be used to capture images of the first object and the second object respectively. For different cameras, images can be captured separately using... Figure 1 or Figure 3 The method shown determines the correspondence between the pixel displacement and the acquisition displacement of the camera. After acquiring a first image using the first camera, the first acquisition displacement can be determined using the correspondence between the pixel displacement of the first camera and the acquisition displacement. After acquiring a second image using the second camera, the second acquisition displacement can be determined using the correspondence between the pixel displacement of the second camera and the acquisition displacement. Therefore, based on the first acquisition displacement, the second acquisition displacement, the position of the first camera, and the position of the second camera, the relative positional relationship between the first object and the second object is determined.

[0115] The image processing method provided in this application can improve the accuracy of positioning.

[0116] The above text combined Figures 1 to 4 The image processing method of the embodiments of this application is described in detail below, and will be combined with Figure 5 and Figure 6 This document describes in detail the apparatus embodiments of this application. It should be understood that the image processing apparatus in the embodiments of this application can execute the various methods described in the foregoing embodiments of this application. That is, the specific working processes of the various products described below can be referred to the corresponding processes in the foregoing method embodiments.

[0117] Figure 5This is a schematic structural diagram of an image processing apparatus provided in an embodiment of this application. The image processing apparatus 500 may include a control unit 510 and a processing unit 520.

[0118] In some embodiments, the image processing apparatus 500 can be used to implement Figure 4 The steps of the method shown.

[0119] The control unit 510 is used to control the first camera to move in a first plane perpendicular to the orientation of the first camera and to acquire images, wherein the first image acquired by the first camera contains a first object.

[0120] The processing unit 520 is used to determine the first acquisition displacement corresponding to the first pixel displacement based on the correspondence between pixel displacement and acquisition displacement, wherein the first pixel displacement is the displacement of the first pixel position of the first object in the first image relative to the center of the first image.

[0121] The processing unit 520 is configured to determine the first position of the first object based on the first acquisition displacement and the first camera position where the first camera was acquiring the first image.

[0122] The correspondence is determined based on a detection data set, which includes the detection pixel displacement and detection acquisition displacement corresponding to each of multiple preset regions. The detection pixel displacement corresponding to any preset region represents the displacement of the detected object's position relative to the center of the second detection image in that preset region. The detection acquisition displacement corresponding to any preset region represents the displacement of the third camera's position relative to the first position of the third camera when acquiring the second detection image of the detected object located in that preset region, compared to the first position of the third camera when acquiring the first detection image. The multiple second detection images and the first detection image of the detected object located in the multiple preset regions are all acquired by the third camera during its movement along a third plane. The third plane is perpendicular to the orientation of the third camera. The distance between the third plane and the detected object is a first distance, and the distance between the first plane and the first object is a second distance. The first distance and the second distance are equal. The third camera and the first camera are the same camera.

[0123] In other embodiments, the image processing device 500 can be used to implement... Figure 1 or Figure 3 The steps of the method shown.

[0124] The control unit 510 is used to control the third camera to move in a third plane perpendicular to the orientation of the third camera, and to control the third camera to acquire a first detection image and multiple second detection images, wherein the detection object recorded in the first detection image is located at the center of the first detection image, and the detection object in different second detection images is located in different preset areas.

[0125] The processing unit 520 is used to determine the correspondence between pixel displacement and acquisition displacement based on the detection data set. The detection data set includes the detection pixel displacement and detection acquisition displacement corresponding to each of the plurality of preset regions. The detection pixel displacement corresponding to any preset region represents the displacement of the position of the detected object in the second detection image of the detected object located in the second detection image of the second detection image of the second detection image of the second detection image of the second detection image of the detected object located in the second detection image of the second detection image of the second detection image of the detected object located in the second detection image of the second detection image of the second detection image of the second detection image of the detected object located in the second detection image of the second detection image of the second detection image of the third camera relative to the first position of the third camera when the first detection image was acquired.

[0126] The correspondence is used to determine the first acquisition displacement corresponding to the first pixel displacement. The first pixel displacement is the displacement of the first pixel position of the first object in the first image acquired by the first camera relative to the center of the first image. The first image is acquired by the first camera while moving on a first plane perpendicular to the orientation of the first camera. The first acquisition displacement and the position of the first camera when acquiring the first image are used to determine the first position of the first object.

[0127] It should be noted that the image processing device 500 described above is embodied in the form of a functional unit. The term "unit" here can be implemented in software and / or hardware, and there is no specific limitation on this.

[0128] For example, a "unit" can be a software program, a hardware circuit, or a combination of both that implements the above functions. The hardware circuit may include an application-specific integrated circuit (ASIC), electronic circuitry, a processor (e.g., a shared processor, a proprietary processor, or a group processor) and memory for executing one or more software or firmware programs, integrated logic circuitry, and / or other suitable components that support the described functions.

[0129] Therefore, the units of the various examples described in the embodiments of this application can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0130] Figure 6 This is a schematic diagram of the structure of an electronic device 600 provided in an embodiment of this application. For example... Figure 6 As shown, the electronic device 600 of this embodiment includes: at least one processor 601 ( Figure 6 The diagram shows only one processor, a memory 602, and a computer program 603 stored in the memory 602 and executable on the at least one processor 601, wherein the processor 601 executes the computer program 603 to implement the steps of any of the above method embodiments.

[0131] Those skilled in the art will understand that Figure 6 This is merely an example of electronic device 600 and does not constitute a limitation on electronic device 600. It may include more or fewer components than shown, or combine certain components, or different components, such as input / output devices, network access devices, etc.

[0132] The processor 601 can be a Central Processing Unit (CPU), but it can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.

[0133] In some embodiments, the memory 602 may be an internal storage unit of the electronic device 600, such as a hard disk or memory of the electronic device 600. In other embodiments, the memory 602 may be an external storage device of the electronic device 600, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the electronic device 600. Furthermore, the memory 602 may include both internal and external storage units of the electronic device 600. The memory 602 is used to store the operating system, applications, bootloader, data, and other programs, such as the program code of the computer program. The memory 602 can also be used to temporarily store data that has been output or will be output.

[0134] It should be noted that the information interaction and execution process between the above-mentioned devices / units are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, and they will not be repeated here.

[0135] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0136] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements any of the above-described method embodiments.

[0137] This application provides a computer program product that, when running, can implement any of the above-described method embodiments.

[0138] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of this application can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include at least: any entity or device capable of carrying the computer program code to a photographing device / terminal device, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks. In some possible implementations, the computer-readable medium cannot be an electrical carrier signal or a telecommunication signal.

[0139] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0140] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0141] In the embodiments provided in this application, it should be understood that the disclosed systems and methods can be implemented in other ways. For example, the system embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interfaces, devices, or units, and may be electrical, mechanical, or other forms.

[0142] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0143] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. An image processing method, characterized in that, The method includes: The first camera is controlled to move on a first plane perpendicular to the orientation of the first camera and to acquire images. The first image acquired by the first camera contains a first object. Based on the correspondence between pixel displacement and acquisition displacement, the first acquisition displacement corresponding to the first pixel displacement is determined. The first pixel displacement is the displacement of the first pixel position of the first object in the first image relative to the center of the first image. The first position of the first object is determined based on the first acquisition displacement and the position of the first camera when the first camera acquires the first image; The correspondence is determined based on a detection data set, which includes the detection pixel displacement and detection acquisition displacement corresponding to each of multiple preset regions. The detection pixel displacement corresponding to any preset region represents the displacement of the detected object's position relative to the center of the second detection image within that preset region. The detection acquisition displacement corresponding to any preset region represents the displacement of the third camera's position relative to the first position of the third camera when acquiring the second detection image of the detected object located within that preset region. The multiple second detection images and the first detection image of the detected object located in the multiple preset regions are all acquired by the third camera during its movement along a third plane. The third plane is perpendicular to the orientation of the third camera. The distance between the third plane and the detected object is a first distance, and the distance between the first plane and the first object is a second distance. The first distance and the second distance are equal. The third camera and the first camera are the same camera.

2. The method according to claim 1, characterized in that, The correspondence is determined through a first operation, which includes: Based on the detection data set, determine the initial correspondence between pixel displacement and acquisition displacement; Based on the initial correspondence, the predicted acquisition displacement corresponding to each detected pixel displacement in the detection data set is determined, and the predicted acquisition displacement corresponding to any detected pixel displacement corresponds to the same preset area as any detected pixel displacement. If the difference between the detected acquisition displacement and the predicted acquisition displacement corresponding to each preset region in the detection data set is less than the difference threshold, the initial correspondence is determined to be the correspondence. When at least one target region exists among the plurality of preset regions, a second operation is performed, wherein the difference between the detected acquisition displacement and the predicted acquisition displacement corresponding to any one of the at least one target region is greater than or equal to a difference threshold. The second operation includes: For any target area, the third camera is controlled to move on the third plane to a position relative to the first position that is the predicted acquisition displacement corresponding to the target area, and a third detection image corresponding to the target area is acquired. The detection acquisition displacement corresponding to any target region in the detection data set is updated to the predicted acquisition displacement corresponding to any target region, and the detection pixel displacement corresponding to any target region is updated to the displacement of the position of the detected object in the third detection image corresponding to any target region relative to the center of the third detection image; After updating the detection data set based on each target region, the correspondence is determined according to the updated detection data set.

3. The method according to claim 2, characterized in that, After updating the detection data set based on each target region, determining the correspondence based on the updated detection data set includes: After updating the detection data set based on each target region, the initial correspondence is re-determined according to the updated detection data set, and the predicted acquisition displacement corresponding to the displacement of each detected pixel in the updated detection data set is determined again, until the difference between the detection acquisition displacement and the predicted acquisition displacement corresponding to each preset region in the updated detection data set is less than the difference threshold.

4. The method according to any one of claims 1-3, characterized in that, The multiple preset regions include multiple regions obtained by uniformly dividing the image captured by the third camera, excluding the region where the center of the image is located.

5. The method according to any one of claims 1-3, characterized in that, The correspondence includes a first proportional relationship between the pixel displacement and the acquisition displacement along a first direction, and a second proportional relationship between the pixel displacement and the acquisition displacement along a second direction, wherein the first direction is perpendicular to the second direction.

6. The method according to any one of claims 1-3, characterized in that, Determining the first position of the first object based on the first acquired displacement includes: Control the first camera to move according to the first acquisition displacement, and acquire the first verification image; Based on the correspondence, the verification acquisition displacement corresponding to the verification pixel displacement is determined, wherein the verification pixel displacement is the displacement of the verification pixel position of the first object in the verification image relative to the center of the verification image; The first position is determined based on the verified displacement.

7. The method according to any one of claims 1-3, characterized in that, The method further includes: The second camera is controlled to move on the first plane so that the second image captured by the second camera records the second object. The orientation of the second camera is the same as that of the first camera, and the third distance between the second object and the first plane is equal to the first distance. Based on the correspondence between pixel displacement and acquisition displacement, the second acquisition displacement corresponding to the second pixel displacement is determined. The second pixel displacement is the displacement of the second pixel position of the second object in the second image relative to the center of the first image. Determining the first position of the first object based on the first acquisition displacement and the position of the first camera when the first camera acquired the first image includes: Based on the first acquisition displacement, the second acquisition displacement, the first camera position, and the second camera position, the relative positional relationship between the first object and the second object is determined, and the second camera position is the position where the second camera is located when acquiring the second image.

8. An image processing method, characterized in that, The method includes: The third camera is controlled to move in a third plane perpendicular to the orientation of the third camera, and the third camera is controlled to acquire a first detection image and multiple second detection images, wherein the detection object recorded in the first detection image is located at the center of the first detection image, and the detection object in different second detection images is located in different preset areas; Based on the detection data set, the correspondence between pixel displacement and acquisition displacement is determined. The detection data set includes the detection pixel displacement and detection acquisition displacement corresponding to each of multiple preset regions. The detection pixel displacement corresponding to any preset region represents the displacement of the position of the detected object in the second detection image of the second detection image relative to the center of the second detection image. The detection acquisition displacement corresponding to any preset region represents the detection acquisition displacement of the position of the third camera when acquiring the second detection image of the detected object located in the second detection image of the second detection image of the second detection image relative to the first position of the third camera when acquiring the first detection image. The correspondence is used to determine the first acquisition displacement corresponding to the first pixel displacement. The first pixel displacement is the displacement of the first pixel position of the first object in the first image acquired by the first camera relative to the center of the first image. The first image is acquired by the first camera while moving on a first plane perpendicular to the orientation of the first camera. The first acquisition displacement and the position of the first camera when acquiring the first image are used to determine the first position of the first object. The distance between the third plane and the detected object is the first distance, and the distance between the first plane and the first object is the second distance. The first distance and the second distance are equal. The third camera is the same camera as the first camera.

9. The method according to claim 8, characterized in that, The step of determining the correspondence between pixel displacement and acquisition displacement based on the detection data set includes: Based on the detection data set, determine the initial correspondence between pixel displacement and acquisition displacement; Based on the initial correspondence, the predicted acquisition displacement corresponding to each detected pixel displacement in the detection data set is determined, and the predicted acquisition displacement corresponding to any detected pixel displacement corresponds to the same preset area as any detected pixel displacement. If the difference between the detected acquisition displacement and the predicted acquisition displacement corresponding to each preset region in the detection data set is less than the difference threshold, the initial correspondence is determined to be the correspondence. When at least one target region exists among the plurality of preset regions, a second operation is performed, wherein the difference between the detected acquisition displacement and the predicted acquisition displacement corresponding to any one of the at least one target region is greater than or equal to a difference threshold. The second operation includes: For any target area, the third camera is controlled to move on the third plane to a position relative to the first position that is the predicted acquisition displacement corresponding to the target area, and a third detection image corresponding to the target area is acquired. The detection acquisition displacement corresponding to any target region in the detection data set is updated to the predicted acquisition displacement corresponding to any target region, and the detection pixel displacement corresponding to any target region is updated to the displacement of the position of the detected object in the third detection image corresponding to any target region relative to the center of the third detection image; After updating the detection data set based on each target region, the correspondence is determined according to the updated detection data set.

10. An electronic device, characterized in that, The device includes a processor and a memory, the memory being used to store a computer program, and the processor being used to retrieve and run the computer program from the memory, causing the electronic device to perform the method of any one of claims 1 to 9.

Images