Mark point matching method and related device

Abstract

The embodiment of the invention provides a mark point matching method and a related device. The method comprises the following steps: acquiring mark information of a mark point; if it is judged that the mark information comprises scale factor information and image coordinate information, determining a first matching relation according to a scale factor; determining a first target matching relationship according to the image coordinate information and the first matching relationship; and according to the first target matching relationship, determining the coding of the first mark point in the mark point image of the current frame in the time sequence. When a mark point with a large displacement is measured on a mobile camera platform, matching of the mark point is realized by matching the scale factors of the mark point in an image, and the matching relation of the scale factors is verified according to the motion constraint through image coordinate information, so that the accuracy of measurement of the mark point is improved. And the accuracy of image measurement is improved.

Description

Technical Field

[0001] The present application belongs to the field of machine vision technology, and specifically relates to a landmark point matching method and related devices. Background Art

[0002] In the field of visual image measurement, matching the marker points in the image taken by the camera is an indispensable part. Only the correct corresponding matching relationship can solve the correct camera platform posture, the displacement of the marker point or the three-dimensional coordinates, etc. Among the commonly used matching methods, the most common one is the coding matching method, which is to encode and mark different marker points, such as: ring coding, and identify the coding mark of the point while extracting the feature point to achieve the matching of the marker point. However, coding recognition has high requirements for image quality. If the camera has a large displacement to produce motion blur or the marker point exceeds the depth of field, the point and line features of the coded marker point will become blurred, resulting in incorrect recognition of the coding mark, which in turn leads to poor accuracy of image measurement. Another is a matching method based on feature points, such as: SIFT (Scale Invariant Feature Transform, scale invariant feature transform matching) matching method, which matches the feature points of two views by detecting feature points in the image, such as corner points. However, this method relies on the texture information in the scene, and may mismatch when there are many repeated textures in the scene, thereby affecting the accuracy of image measurement. Summary of the invention

[0003] The embodiments of the present application provide a marker point matching method and related devices, which can achieve matching of marker points by matching the scale factors of marker points in an image when measuring on a mobile camera platform or measuring marker points with large displacements, and verify the matching relationship of the scale factors according to motion constraints through image coordinate information, which is beneficial to improving the accuracy of image measurement.

[0004] In a first aspect, an embodiment of the present application provides a landmark matching method, which is applied to a processing device in an image measurement system, wherein the image measurement system includes the processing device, a camera platform, a landmark, and a camera arranged on the camera platform, wherein the processing device is connected to the camera, and the landmark includes a first sub-marker and a second sub-marker, wherein the first sub-marker and the second sub-marker have a scale feature and a direction feature; the method includes: Acquire marker information of the marker point, wherein the marker information includes scale factor information and / or image coordinate information and / or coding mark information of the marker point in multiple frames of marker point images and / or first mileage information of the camera platform; If it is determined that the mark information includes the scale factor information and the image coordinate information, determining a first matching relationship between a first scale factor of a first mark point in a mark point image of a current frame and a second scale factor of a second mark point in a mark point image of a previous frame according to the scale factor information; Determine a first target matching relationship between a first marker point in the marker point image of the current frame and a second marker point in the marker point image of the previous frame according to the image coordinate information and the first matching relationship; According to the first target matching relationship, the temporal encoding of the first marker point in the marker point image of the current frame is determined.

[0005] In a possible example, determining, according to the scale factor information, a first matching relationship between a first scale factor of a first marker point in a current frame marker point image and a second scale factor of a second marker point in a previous frame marker point image includes: Perform the following operation on the first scale factor of each first marker point in the marker point image of the current frame to obtain the first matching relationship: Determine the difference between the first scale factor currently being processed and each second scale factor to obtain a plurality of difference values; determine the absolute values ​​of the plurality of difference values ​​to obtain a plurality of target values; Determine a target second proportional factor corresponding to the smallest value among the multiple target values; If it is determined that the target value corresponding to the target second scale factor is less than the first preset threshold, it is determined that the currently processed first scale factor has a matching relationship with the target second scale factor.

[0006] In a possible example, determining, according to the image coordinate information and the first matching relationship, a first target matching relationship between a first marker point in the current frame marker point image and a second marker point in the previous frame marker point image includes: Perform the following operation on each set of mutually matching first proportional factors and second proportional factors in the first matching relationship to determine a first target matching relationship between the first marker point and the second marker point: Determine, according to the image coordinate information, a conversion relationship between a first marker point corresponding to a first scale factor currently being processed and a second marker point corresponding to a second scale factor currently being processed; According to the conversion relationship, construct a target equation group; Solving the target equation group to obtain a rotation matrix; According to the rotation matrix, a reprojection error is estimated to obtain a target reprojection error; If it is determined that the target reprojection error is less than or equal to the sixth preset threshold, it is determined that a first marker point corresponding to the currently processed first scale factor and a second marker point corresponding to the currently processed second scale factor have a matching relationship.

[0007] In a possible example, after acquiring the mark information of the mark point, the method further includes: if it is determined that the mark information includes the scale factor information, the image coordinate information and the first mileage information, predicting the mileage prediction information and the scale factor prediction information of the mark point image of the current frame according to the second scale factor of the second mark point in the mark point image of the previous frame, the first mileage information and a preset prediction model; Determine, according to the scale factor prediction information and the first scale factor of the marker point image of the current frame, a second matching relationship between a third scale factor in the scale factor prediction information and the first scale factor of the marker point image of the current frame; Correcting the preset prediction model according to the second matching relationship to obtain a corrected preset prediction model; Determine, according to the scale factor information, a first distance from each marker point in the multiple-frame marker point images to the optical center of the camera; Determining second mileage information according to the first distance; Determine first position information of the marker point in the multiple-frame marker point image according to the second mileage information and the scale factor information; Determine second position information of the first marker point according to the first mileage information and the scale factor information; The first position information and the second position information are matched to obtain a second target matching relationship of the marker point position.

[0008] In a possible example, after acquiring the marker information of the marker point, the method further includes: If it is determined that the mark information includes the scale factor information, the image coordinate information and the encoding mark information, determining a first target matching relationship according to the scale factor information and the image coordinate information; Determine image quality information for each landmark point; Filter out target landmark points whose image quality information meets preset image conditions; Determining first absolute coding information of the target marker point according to the coding mark information; According to the first target matching relationship and the first absolute coding information, second absolute coding information of the marker point in the marker point image of the current frame is determined.

[0009] In a possible example, after acquiring the marker information of the marker point, the method further includes: If it is determined that the mark information includes the scale factor information, the image coordinate information, the first mileage information and the coded mark information, determining a second target matching relationship according to the scale factor, the image coordinate information and the first mileage information; The second absolute coding information is determined according to the scale factor information, the image coordinate information and the coding mark information.

[0010] In a possible example, before acquiring the marker information of the marker point, the method further includes: The following operation is performed on each first marker point to obtain the first scale factor of each first marker point: Determine a physical dimension value between a first sub-marker and a second sub-marker of a first marker point currently being processed; Determine a pixel scale value between a first sub-marker and a second sub-marker of the first marker point currently being processed imaged in the marker point image of the current frame; A first scale factor of the first marker point currently being processed is determined according to the physical scale value and the pixel scale value.

[0011] In a possible example, after determining the temporal encoding of the first marker point in the marker point image of the current frame according to the first target matching relationship, the method further includes: If it is detected that there is a first target marker point that does not match the second marker point in the marker point image of the current frame, determine a fourth scale factor with the smallest scale factor value and a fifth scale factor with the largest scale factor value in the marker point image of the previous frame; Determine a difference between a first scale factor corresponding to the first target landmark point and the fourth scale factor to obtain a first target difference; Determine a difference between a first scale factor corresponding to the first target landmark and the fifth scale factor to obtain a second target difference; If it is determined that the first target difference is less than the second preset threshold or the second target difference is greater than the third preset threshold, the encoding of the first target marker point is determined based on the order of appearance of the first target marker points and the existing encoding information of the first marker point in the current frame marker point image.

[0012] In a second aspect, an embodiment of the present application provides a marker point matching device, which is applied to a processing device in an image measurement system, wherein the image measurement system includes the processing device, a camera platform, a marker point, and a camera arranged on the camera platform, wherein the processing device is connected to the camera, wherein the marker point includes a first sub-marker and a second sub-marker, wherein the first sub-marker and the second sub-marker have a scale feature and a direction feature; wherein the device includes an acquisition unit and a determination unit; wherein, The acquisition unit is used to acquire the mark information of the mark point, wherein the mark information includes the scale factor information and / or the image coordinate information and / or the coding mark information and / or the first mileage information of the camera platform of the mark point in the multiple frames of the mark point image; The determining unit is configured to determine, if it is determined that the marker information includes the scale factor information and the image coordinate information, a first matching relationship between a first scale factor of a first marker point in a marker point image of a current frame and a second scale factor of a second marker point in a marker point image of a previous frame according to the scale factor information; The determining unit is further configured to determine a first target matching relationship between a first marker point in the marker point image of the current frame and a second marker point in the marker point image of the previous frame according to the image coordinate information and the first matching relationship; The determining unit is further configured to determine the temporal encoding of the first marker point in the marker point image of the current frame according to the first target matching relationship.

[0013] A third aspect of the present application provides an electronic device, comprising: a processor and a memory; and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the processor, and the programs include instructions for some or all of the steps described in the first aspect.

[0014] A fourth aspect of an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium is used to store a computer program, wherein the computer program enables a computer to execute instructions of some or all of the steps described in the first aspect of the embodiment of the present application.

[0015] A fifth aspect of the embodiments of the present application provides a computer program product, wherein the computer program product includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to cause a computer to execute some or all of the steps described in the first aspect of the embodiments of the present application. The computer program product may be a software installation package.

[0016] It can be seen that in the embodiment of the present application, the processing device can first obtain the mark information of the mark point, the mark information includes the scale factor information and / or image coordinate information and / or the coding mark information and / or the first mileage information of the camera platform of the mark point in the multiple frames of the mark point image, and then if it is determined that the mark information includes the scale factor information and the image coordinate information, then according to the scale factor, determine the first matching relationship between the first scale factor of the first mark point in the current frame of the mark point image and the second scale factor of the second mark point in the previous frame of the mark point image, and then according to the image coordinate information and the first matching relationship, determine the first target matching relationship between the first mark point in the current frame of the mark point image and the second mark point in the previous frame of the mark point image, and finally, according to the first target matching relationship, determine the temporal encoding of the first mark point in the current frame of the mark point image. It can be achieved that when measuring on a mobile camera platform or measuring a mark point with a large displacement, the scale factor of the mark point in the image is matched, and then the mark point is matched, and the matching relationship of the scale factor is verified according to the motion constraint through the image coordinate information, which is conducive to improving the accuracy of image measurement. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0018] Figure 1 is a schematic diagram of the architecture of an image measurement system provided in an embodiment of the present application; Figure 2 It is a flowchart of a landmark point matching method provided in an embodiment of the present application; Figure 3 is a schematic diagram of a landmark point provided in an embodiment of the present application; Figure 4 is a schematic diagram of another marking point provided in an embodiment of the present application; Figure 5 is a schematic diagram of an object plane, an image plane and an optical center provided in an embodiment of the present application; Figure 6 is a schematic diagram of a marker point and a camera provided in an embodiment of the present application; Figure 7 is a schematic diagram of a simulation curve provided in an embodiment of the present application; Figure 8 is a schematic diagram of another simulation curve provided in an embodiment of the present application; Fig. 9It is a structural schematic diagram of an electronic device provided in an embodiment of the present application; Fig.10 It is a block diagram of the functional units of a landmark point matching device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0019] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0020] The terms "first", "second", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally includes steps or units that are not listed, or optionally includes other steps or units inherent to these processes, methods, products or devices.

[0021] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0022] In the embodiments of the present application, "and / or" describes the association relationship of the associated objects, indicating that three relationships may exist. For example, A and / or B can represent the following three situations: A exists alone; A and B exist at the same time; B exists alone. Among them, A and B can be singular or plural.

[0023] In the embodiment of the present application, the symbol " / " can indicate that the objects associated with each other are in an "or" relationship. In addition, the symbol " / " can also indicate a division sign, that is, performing a division operation. For example, A / B can indicate A divided by B.

[0024] In the embodiments of the present application, "at least one item" or similar expressions refer to any combination of these items, including any combination of single items or plural items, and refer to one or more, and multiple refers to two or more. For example, at least one item of a, b, or c can represent the following seven situations: a, b, c, a and b, a and c, b and c, a, b, and c. Among them, each of a, b, and c can be an element or a set containing one or more elements.

[0025] In the embodiments of the present application, "equal to" can be used in conjunction with greater than, and is applicable to the technical solution adopted when greater than, and can also be used in conjunction with less than, and is applicable to the technical solution adopted when less than. When equal to is used in conjunction with greater than, it is not used in conjunction with less than; when equal to is used in conjunction with less than, it is not used in conjunction with greater than.

[0026] In order to better understand the solutions of the embodiments of the present application, the electronic devices, related concepts and backgrounds that may be involved in the embodiments of the present application are first introduced below.

[0027] The electronic device of the embodiment of the present application is a device with a wireless communication function, which can be called a terminal, user equipment (UE), mobile station (MS), mobile terminal (MT), access terminal equipment, vehicle-mounted terminal equipment, industrial control terminal equipment, UE unit, UE station, mobile station, remote station, remote terminal equipment, mobile equipment, UE terminal equipment, wireless communication equipment, UE agent or UE device, etc. The terminal device can be fixed or mobile. It should be noted that the terminal device can support at least one wireless communication technology, such as LTE, new radio (NR), wideband code division multiple access (WCDMA), etc. For example, the terminal device may be a mobile phone, a tablet computer (pad), a desktop computer, a laptop computer, an all-in-one computer, a vehicle-mounted terminal, a virtual reality (VR) terminal device, an augmented reality (AR) terminal device, a wireless terminal in industrial control, a wireless terminal in self driving, a wireless terminal in remote medical surgery, a wireless terminal in a smart grid, a wireless terminal in transportation safety, a wireless terminal in a smart city, a wireless terminal in a smart home, a cellular phone, a cordless phone, a session initiation protocol (SIP) phone, a wireless local loop (WLL) station, a personal digital assistant (PDA), a handheld device with wireless communication function, an electronic device or other processing device connected to a wireless modem, a wearable device, a terminal device in a future mobile communication network, or a terminal device in a future evolved public mobile land network (PLMN), etc. The electronic device may be a processing device in an image measurement system.

[0028] See also Figure 1 , Figure 1 Schematic diagram of the architecture of an image measurement system provided in an embodiment of the present application. Figure 1As shown, the image measurement system 1 includes a processing device 10, a camera platform 20, a camera 30, and a marker 40, and the processing device 10 and the camera 30 are connected.

[0029] The processing device 10 may be a server or a processor, which is not limited here.

[0030] Among them, the camera platform 20 can be a mobile platform, such as: a vehicle, a drone, an unmanned boat and other carriers, the camera 30 is fixed on the mobile platform to shoot the stationary mark point 40; the camera platform 20 can also be a fixed platform, the camera 30 is set on the fixed platform to shoot the moving mark point 40, for example: when the bridge structure is used as the structure to be tested, the jacking method is used to build the bridge, and the beam structure needs to be jacked to the design. During this process, the beam structure will undergo a large displacement; for example: when structures such as conveyor belts are used as structures to be tested, the conveyor belts are moving during operation.

[0031] Among them, the marker point 40 has physical structural information, which can be a natural marker or an artificial marker, which is not limited here. For example: natural markers include but are not limited to lane lines on roads, rails, sleepers, spikes on railways, etc.; artificial markers include but are not limited to luminous light source markers, reflective markers, regular patterns of flat plates, etc. Physical structural information includes not only the point and line features of the marker point 40 with geometric relationships, but also the shape features of each part of the marker point 40, etc., where points include but are not limited to corner points, centroids, centroids, etc., lines include but are not limited to straight lines, curves, dotted lines, etc., and shapes include but are not limited to circles, ellipses, polygons, etc. The point and line features and shape features do not change their relative positions as the position and posture of the marker point 40 change, that is, the marker point 40 always maintains fixed physical structural information during the measurement process. The geometric relationship formed by these point and line features and shape features can be used to find structural features with scale information. Information with scale characteristics includes, but is not limited to, the distance between points, the distance between points and lines, the distance between lines, the scale of lines, the size of shapes, the distance between the centroids of shapes, etc., such as the length and width of a highway lane line, the length and width of a railway sleeper, the distance between the center points of two lane lines, the distance between the center points of two spikes, etc. The aforementioned scale information can be used to calibrate the scale factor in real time. The scale factor refers to the ratio of the image resolution to the physical space resolution. The scale factor is related to the equivalent focal length of the camera 30 and the distance between the camera 30 and the object to be measured, and represents the magnification relationship between the image pixel change value at each object to be measured in the image captured by the camera 30 and the actual physical displacement of the object to be measured.

[0032] For example, the scale information of the marker point 40 to be measured is the distance between the center points of two lane lines. The actual distance between the center points of the two lane lines is D. In the image captured by the camera 30, the pixel scale between the center points of the two lane lines is d, and the scale factor k = d / D. The scale factor can establish the image coordinate change and the real physical displacement The connection between .

[0033] The camera 30 is used to capture images of the marker points 40 and transmit the captured images to the processing device 10 .

[0034] Among them, the image measurement system 1 also includes a device for obtaining the first mileage information, and the first mileage information of the camera platform 20 can be obtained by using a visual odometer or a mobile platform equipped with multi-source sensors, such as an accelerometer, an inertial measurement unit IMU, an inertial navigation system, a global positioning system GPS, an encoded odometer, etc. The first mileage information includes but is not limited to the speed, acceleration, moving distance, moving direction, etc. of the camera platform 20.

[0035] Among them, the landmark point matching system may include a memory connected to the processing device 10, and the memory is used to store the matching relationship between the landmark points 40 in two adjacent frames of landmark point images determined by the processing device 10 and the absolute coding information of the landmark points 40.

[0036] In a possible example, the processing device 10 may first obtain the marking information of the marking point 40, where the marking information includes the scale factor information and / or image coordinate information and / or the coded mark information and / or the first mileage information of the marking point 40 in the multi-frame marking point images. Then, if the processing device 10 determines that the marking information includes the scale factor information and the image coordinate information, then, based on the scale factor, the processing device 10 determines a first matching relationship between a first scale factor of a first marking point in a current frame marking point image and a second scale factor of a second marking point in a previous frame marking point image. Then, based on the image coordinate information and the first matching relationship, the processing device 10 determines a first target matching relationship between the first marking point in the current frame marking point image and the second marking point in the previous frame marking point image. Finally, the processing device 10 determines the temporal encoding of the first marking point in the current frame marking point image based on the first target matching relationship. When measuring on a mobile camera platform 20 or measuring a mark point 40 with a large displacement, the scale factor of the mark point 40 in the image is matched to achieve matching of the mark point 40, and the matching relationship of the scale factor is verified according to the motion constraint through the image coordinate information, which is beneficial to improving the accuracy of image measurement.

[0037] See also Figure 2 , Figure 2 It is a flow chart of a landmark matching method provided in an embodiment of the present application, which is applied to a processing device in an image measurement system, wherein the image measurement system comprises the processing device, a camera platform, a landmark, and a camera arranged on the camera platform, wherein the processing device is connected to the camera, wherein the landmark comprises a first sub-marker and a second sub-marker, wherein the first sub-marker and the second sub-marker have a scale feature and a direction feature, and wherein the method comprises: Step S201, obtaining marker information of a marker point, wherein the marker information includes scale factor information and / or image coordinate information and / or coding mark information and / or first mileage information of the marker point in multiple frames of marker point images.

[0038] Among them, the scale features include but are not limited to the distance between points, the distance between points and lines, the distance between lines, the scale of lines, the size of shapes, the distance between the centroids of shapes, etc., which are not limited here. The points include but are not limited to corner points, centroids, centroids, centers, etc., lines include but are not limited to straight lines, curves, dotted lines, etc., and shapes include but are not limited to circles, ellipses, polygons, irregular figures, etc. For example, the first sub-marker and the second sub-marker are circles, and the first scale feature is that the distance between the center of the first sub-marker and the center of the second sub-marker is equal to the preset distance. The preset distance can be set manually or by the system default, which is not limited here. The diameter of the first sub-marker is the first preset value, and the diameter of the second sub-marker is the second preset value. The first preset value and the second preset value can be set manually or by the system default, which are not limited here.

[0039] See also Figure 3 , Figure 3 It is a schematic diagram of a marker point provided in an embodiment of the present application, wherein the marker point includes a first sub-marker and a second sub-marker, wherein the first sub-marker and the second sub-marker are circles, and the scale information of the marker point includes that the length of the marker point is 98 mm, the width is 60 mm, the diameter of the first sub-marker is 50 mm, the diameter of the second sub-marker is 34 mm, the distance from the center of the second sub-marker to the bottom edge of the marker point is 27 mm, the distance from the center of the first sub-marker to the center of the second sub-marker is 55 mm, and a coding mark is added in the first sub-marker.

[0040] The directional feature refers to the arrangement direction of the first sub-marker and the second sub-marker, which is consistent with the direction of the scale factor calibration of the marker point. For example, when the scale factor in the vertical direction needs to be calibrated, the arrangement direction is the vertical direction. Figure 4 , Figure 4This is a schematic diagram of another type of marker point provided in an embodiment of the present application. The arrangement directions of the first sub-marker and the second sub-marker can be spread across all directions, so that the proportional factors of multiple directions can be calibrated at one time.

[0041] The multiple-frame marker point images include a current-frame marker point image and a historical-frame marker point image. The historical-frame marker point image includes a previous-frame marker point image. The marker point image is an image of the marker point taken by a camera.

[0042] Among them, a single landmark point corresponds to a proportional factor, which can be calibrated in real time.

[0043] The image coordinate information includes the coordinates of each marker point in the image coordinate system.

[0044] Among them, the coded mark information includes the coded mark of the marker point, the scale information between the first sub-marker and the second sub-marker, the length and width of the marker point, the distance from the centroid of the first sub-marker to the bottom and side of the marker point, the distance from the centroid of the second sub-marker to the bottom and side of the marker point, etc.

[0045] The first mileage information includes the speed, acceleration, moving distance, moving direction, etc. of the camera platform when each frame of image is captured.

[0046] Step S202: If it is determined that the mark information includes the scale factor information and the image coordinate information, a first matching relationship between a first scale factor of a first mark point in a mark point image of a current frame and a second scale factor of a second mark point in a mark point image of a previous frame is determined according to the scale factor information.

[0047] For example: the scale factor calibration direction of the marker point is the vertical direction. The first frame of the marker point image includes three marker points, which appear from top to bottom. The three marker points are coded as 1, 2 and 3 in the order of appearance. The second frame of the marker point image includes four marker points. The fourth marker point appears below the marker point image. The code of the fourth marker point is 4. The four marker points are coded as 1, 2, 3 and 4 in the order of appearance.

[0048] Step S203: determining a first target matching relationship between a first marker point in the marker point image of the current frame and a second marker point in the marker point image of the previous frame according to the image coordinate information and the first matching relationship.

[0049] Among them, proximity matching is performed according to the size of the scale factor.

[0050] Step S204: Determine the temporal encoding of the first marker point in the marker point image of the current frame according to the first target matching relationship.

[0051] After determining the temporal encoding of the first marker point in the current frame marker point image according to the first target matching relationship, the following steps are also included: if it is detected that there is a first target marker point in the current frame marker point image that is not matched with the second marker point, then determine the fourth scale factor with the smallest scale factor value and the fifth scale factor with the largest scale factor value in the previous frame marker point image; determine the difference between the first scale factor corresponding to the first target marker point and the fourth scale factor to obtain a first target difference; determine the difference between the first scale factor corresponding to the first target marker point and the fifth scale factor to obtain a second target difference; if it is determined that the first target difference is less than the second preset threshold or the second target difference is greater than the third preset threshold, then determine the encoding of the first target marker point according to the appearance order of the first target marker points and the existing encoding information of the first marker point in the current frame marker point image.

[0052] For example, the current frame marker point image includes five marker points, and the scale factors are 1, 8, 20, 63 and 120 respectively. The previous frame marker point image includes four marker points, and the codes are 4, 5, 6 and 7 respectively. The scale factors are 2, 9, 21 and 64 respectively. The first preset threshold is 5, the second preset threshold is -3, and the third preset threshold is 20. Then the first matching relationship of the scale factor is as follows: according to the condition that the absolute value of the scale factor difference is less than the first preset threshold, the marker point with a scale factor of 1 in the current frame marker point image matches the marker point with a scale factor of 2 in the previous frame marker point image, and the marker point with a scale factor of 1 in the current frame marker point image is encoded as 4; the marker point with a scale factor of 8 in the current frame marker point image matches the marker point with a scale factor of 9 in the previous frame marker point image, and the marker point with a scale factor of 8 in the current frame marker point image is encoded as 5 ; The marker point with a scale factor of 20 in the current frame marker point image matches the marker point with a scale factor of 21 in the previous frame marker point image, and the marker point with a scale factor of 20 in the current frame marker point image is encoded as 6; the marker point with a scale factor of 63 in the current frame marker point image matches the marker point with a scale factor of 64 in the previous frame marker point image, and the marker point with a scale factor of 63 in the current frame marker point image is encoded as 7. The marker point with a scale factor of 120 in the current frame marker point image (the first target marker point) has no corresponding matching second marker point. Since the second target difference between the marker point with a scale factor of 120 in the current frame marker point image and the maximum scale factor of 64 (the fifth scale factor) in the previous frame marker point image is greater than the third preset threshold, that is, 120-64>20, the marker point with a scale factor of 120 in the current frame marker point image is regarded as a newly appeared point and is encoded as 8.

[0053] Another possible example, the current frame marker point image includes four marker points, and the scale factors are 2, 9, 21 and 64 respectively, the previous frame marker point image includes three marker points, and the codes are 7, 6 and 5 respectively, and the scale factors are 8, 20 and 63 respectively. The first preset threshold is 5, the second preset threshold is -3, and the third preset threshold is 20. Then the first matching relationship of the scale factor is as follows: according to the condition that the absolute value of the scale factor difference is less than the first preset threshold, the marker point with a scale factor of 9 in the current frame marker point image matches the marker point with a scale factor of 8 in the previous frame marker point image, and the marker point with a scale factor of 9 in the current frame marker point image is encoded as 7; the marker point with a scale factor of 21 in the current frame marker point image matches the marker point with a scale factor of 20 in the previous frame marker point image, and the marker point with a scale factor of 21 in the current frame marker point image is encoded as 6 ; The marker point with a scale factor of 64 in the current frame marker point image matches the marker point with a scale factor of 63 in the previous frame marker point image, and the marker point with a scale factor of 64 in the current frame marker point image is encoded as 5. However, the marker point with a scale factor of 2 in the current frame marker point image (the first target marker point) has no corresponding matching second marker point. Since the first target difference between the marker point with a scale factor of 2 in the current frame marker point image and the minimum scale factor of 8 (the fourth scale factor) in the previous frame marker point image is less than the second preset threshold, that is, 2-8<-3, the marker point with a scale factor of 2 in the current frame marker point image is regarded as a newly appeared point and is encoded as 8.

[0054] Among them, the image coordinate information of the marker point is used to determine whether the matching situation is reliable based on the motion constraint. The motion constraint means that in the case of high-speed video recording or low-speed motion, the change of the camera platform between two frames of marker point images is approximately only the angle change, the displacement change can be ignored, and the reprojection error is less than the sixth preset threshold.

[0055] Among them, the first landmark point and the second landmark point that match each other in the first target matching relationship can be considered to be the same landmark point.

[0056] It can be seen that in the embodiment of the present application, the processing device can first obtain the mark information of the mark point, the mark information includes the scale factor information and / or image coordinate information and / or the coding mark information and / or the first mileage information of the camera platform of the mark point in the multiple frames of the mark point image, and then if it is determined that the mark information includes the scale factor information and the image coordinate information, then according to the scale factor, determine the first matching relationship between the first scale factor of the first mark point in the current frame of the mark point image and the second scale factor of the second mark point in the previous frame of the mark point image, and then according to the image coordinate information and the first matching relationship, determine the first target matching relationship between the first mark point in the current frame of the mark point image and the second mark point in the previous frame of the mark point image, and finally, according to the first target matching relationship, determine the temporal encoding of the first mark point in the current frame of the mark point image. It can be achieved that when measuring on a mobile camera platform or measuring a mark point with a large displacement, the scale factor of the mark point in the image is matched, and then the mark point is matched, and the matching relationship of the scale factor is verified according to the motion constraint through the image coordinate information, which is conducive to improving the accuracy of image measurement.

[0057] In one possible example, in terms of determining, based on the scale factor information, a first matching relationship between a first scale factor of a first marker point in a current frame marker point image and a second scale factor of a second marker point in a previous frame marker point image, the method may include the following steps: performing the following operations on the first scale factor of each first marker point in the current frame marker point image to obtain the first matching relationship: determining the difference between the currently processed first scale factor and each second scale factor to obtain a plurality of differences; determining the absolute values ​​of the plurality of differences to obtain a plurality of target values; determining a target second scale factor corresponding to the smallest value among the plurality of target values; if it is determined that the target value corresponding to the target second scale factor is less than a first preset threshold, determining that the currently processed first scale factor has a matching relationship with the target second scale factor.

[0058] Among them, the current frame marker point image has three first marker points, and the scale factors are 9, 21 and 64 respectively. The previous frame marker point image has three second marker points, and the scale factors corresponding to the three marker points are 8, 20, and 63 respectively. The first preset threshold is 1.5, and the multiple differences between the first scale factor of the first marker point with a scale factor of 9 and the second scale factor of each second marker point are 1, -11 and -54 respectively. After taking the absolute values ​​of the multiple differences, the multiple target values ​​obtained are 1, 11 and 54. The smallest value is 1, 1<1.5, and the target second scale factor corresponding to 1 is 8. At this time, there is a matching relationship between the first scale factor 9 and the second scale factor 8. Similarly, it is concluded that the first scale factor 21 matches the second scale factor 20, and the first scale factor 64 matches the second scale factor 63. Then the first matching relationship finally obtained is that the first scale factor 9 matches the second scale factor 8, the first scale factor 21 matches the second scale factor 20, and the first scale factor 64 matches the second scale factor 63.

[0059] The first preset threshold may be manually set or set by system default, and is not limited here.

[0060] It can be seen that in this example, by matching the scale factors of the marker points in the image, the matching of the marker points is achieved, which is beneficial to improving the accuracy of image measurement.

[0061] In one possible example, in terms of determining the first target matching relationship between the first marker point in the current frame marker point image and the second marker point in the previous frame marker point image based on the image coordinate information and the first matching relationship, the above method may include the following steps: performing the following operations on each group of mutually matching first scale factors and second scale factors in the first matching relationship to determine the first target matching relationship between the first marker point and the second marker point: determining the conversion relationship between the first marker point corresponding to the first scale factor currently being processed and the second marker point corresponding to the second scale factor currently being processed based on the image coordinate information; constructing a target set of equations based on the conversion relationship; solving the target set of equations to obtain a rotation matrix; estimating a reprojection error based on the rotation matrix to obtain a target reprojection error; if it is determined that the target reprojection error is less than or equal to a sixth preset threshold, determining that there is a matching relationship between the first marker point corresponding to the first scale factor currently being processed and the second marker point corresponding to the second scale factor currently being processed.

[0062] Assuming that the camera only changes in angle, the change in the camera pose between two frames can be described by the rotation matrix R composed of Euler angles in three directions:

[0063] Assume the normalized coordinates of the matching points in the two images are:

[0064] in, represents the normalized coordinates of the first landmark point, represents the normalized coordinates of the second landmark point, The two coordinates satisfy:

[0065] Expands to:

[0066] Among them, K is the camera intrinsic parameter matrix, which is obtained by camera calibration.

[0067] Each pair of first and second landmark points can provide two equations, and three pairs of first and second landmark points can solve nine parameters. Using at least three pairs of matching points to construct an overdetermined set of equations, namely the target set of equations, the least squares solution is solved by singular value decomposition to obtain the rotation matrix R:

[0068] The first landmark point of the landmark image of the current frame is projected onto the landmark image of the previous frame through the calculated rotation matrix R to obtain the projection point , calculate the projection point With the actual point The Euclidean distance of is used to estimate the reprojection error:

[0069] Among them, Error represents the reprojection error.

[0070] Among them, the sixth preset threshold can be set manually or by system default, and is not limited here.

[0071] If it is determined that the target reprojection error is greater than a sixth preset threshold, it is determined that there is no matching relationship between the first marker point corresponding to the first scale factor currently being processed and the second marker point corresponding to the second scale factor currently being processed.

[0072] It can be seen that in this example, verifying the matching relationship of the scale factor according to the motion constraint through the image coordinate information is conducive to improving the accuracy of image measurement.

[0073] In a possible example, after acquiring the marker information of the marker point, the method may include the following steps: if it is determined that the marker information includes the scale factor information, the image coordinate information and the first mileage information, predicting the mileage prediction information and the scale factor prediction information of the marker point image of the current frame according to the second scale factor of the second marker point in the marker point image of the previous frame, the first mileage information and the preset prediction model; determining a second matching relationship between a third scale factor in the scale factor prediction information and a first scale factor of the marker point image of the current frame according to the scale factor prediction information and the first scale factor of the marker point image of the current frame; correcting the preset prediction model according to the second matching relationship to obtain a corrected preset prediction model; determining a first distance from each marker point in the multiple marker point images to the optical center of the camera according to the scale factor information; determining the second mileage information according to the first distance; determining the first position information of the marker point in the multiple marker point images according to the second mileage information and the scale factor information; determining the second position information of the first marker point according to the first mileage information and the scale factor information; matching the first position information with the second position information to obtain a second target matching relationship of the marker point position.

[0074] Among them, the Kalman filter and other methods can be used to use the preset prediction model to predict the mileage and scale factors of the landmark points of the current frame camera platform, and then the measured value (first scale factor) and the predicted value (third scale factor) of the scale factor of the landmark point image of the current frame are matched in numerical value to obtain the second matching relationship between the first scale factor and the third scale factor. If the current camera frame is the first frame, the Kalman filter is used with the initial state variables and the initial covariance matrix using the preset prediction model based on motion constraints to predict the scale factors of the landmark points of the next frame. If the current camera frame is not the first frame, the first scale factor set of the current frame is The third scale factor set with the prediction Perform proximity matching of the numerical values ​​of the scale factors and correct the state variables and covariance matrix of the current preset prediction model.

[0075] Among them, the relationship between the scale factor and the equivalent focal length is ,in, Represents the distance from the object to the optical center of the camera along the optical axis. Therefore, each time a marker point is detected, the distance from the point to the optical center can be calculated. Subtracting the distances from the corresponding points in the two frames to the optical center can obtain the camera / marker point movement distance between the two frames. At the same time, if the frame rate fps of the camera is known, that is, the time difference between two frames is considered to be 1 / fps, the camera / marker point motion speed of the frame can be obtained as Therefore, each time the landmark matching relationship between two frames is obtained, the movement distance of the camera or landmark can be calculated by the scale factor. And the speed v. In view of the above characteristics that the motion distance and motion speed can be directly obtained from the scale factor, the scale factors of all landmarks in the camera frame are tracked based on the Kalman filter: the state vector Defined as displacement and speed :

[0076] The prediction model is:

[0077] Among them, the state transfer matrix for:

[0078] Among them, the control input matrix for:

[0079] in, is the control input (or external input), which represents the active control quantity of the system (such as applied force, acceleration, etc.). This variable is the external factor that causes the state variable to change. In this embodiment, the state variables are displacement and velocity, so the control input Usually corresponds to acceleration, which can be caused by factors such as accelerator, brake, motor or manual push.

[0080] Among them, process noise Obey Gaussian distribution:

[0081] The observation equation is:

[0082] The observation matrix is: , observation noise Obey Gaussian distribution: , the prediction step of Kalman filter: , , that is, the displacement of the current frame is predicted to be , converting the predicted displacement into a predicted scale factor , and the actual scale factor of the landmark point in the current frame is matched in size. is the state error covariance matrix, and Q is the noise error covariance matrix. After the matching relationship is confirmed, it is assumed that the scale factor of the previous frame mark point is , the scale factor of the current frame landmark is Then the observation equation The displacement is , the speed is . Kalman filter correction steps: , , , with the first frame camera / marker point stationary as the initial value: By repeating the process of predicting the scale factor and correcting the error covariance matrix, the landmark point can be tracked.

[0083] For example, the second scale factor set of the previous frame of the marker image , the predicted third scale factor set , The corresponding prediction is , The corresponding prediction is , The corresponding prediction is , The corresponding prediction is , the first scale factor set of the current frame landmark image , if the result of the proximity match between the third scale factor set and the first scale factor set is, the second matching relationship is and match, and match, and match, is the smallest scale factor in the third scale factor set, is the largest scale factor in the second scale factor set, or ,but The corresponding landmark point is a newly appeared landmark point, which can be time-series encoded.

[0084] The third matching relationship between the first scale factor in the current frame mark point image and the second scale factor of the previous frame mark point can be known from the second matching relationship. For example, the second matching relationship is and match, and match, and Match, then the third matching relationship is and match, and match, and .

[0085] Among them, the relationship between the scale factor and the equivalent focal length is ,in, Represents the distance from the object to the optical center of the camera along the optical axis. Therefore, each time a scale factor of a marker point is detected, the distance from the marker point to the optical center, i.e., the first distance, can be calculated. Subtracting the distances from the corresponding points of the two frames to the optical center, the camera movement distance between the two frames can be obtained. Since multiple landmarks can be observed between two frames, the scale factors of all landmarks observed in the two frames can be used for settlement to obtain multiple camera motion distances. The multiple camera motion distances are optimized by least squares to obtain the best camera motion distance between the two frames, i.e., the second mileage information. For example, the average of multiple camera motion distances can be calculated and used as the best camera motion distance between the two frames.

[0086] The first position information includes the distance of the marker points in all the frame marker point images relative to the first camera frame. The first position information is used as a reference for global matching. The second position information includes the distance of the marker points in the current frame marker point image relative to the first camera frame. Due to the influence of feature point extraction or noise, the third matching relationship has a certain probability of mismatching and even the measured number of marker points does not match the actual number. It is also necessary to use mileage information to further optimize the results of temporal coding matching. The distance of the marker point relative to the first camera frame (Assuming that the nth marker point is observed in the mth frame), similarly, since the same marker point can be observed by multiple camera frames, the multiple distances of the marker points calculated by different camera frames relative to the first camera frame (the mileage of the first camera frame is zero) can be optimized by least squares to obtain the best distance of each marker point relative to the first camera frame, that is, the first position information. For example: the average of multiple distances is taken as the best distance of each marker point relative to the first camera frame.

[0087] For example, the first position information includes marker points 1, 2, 3, 4, and 5, and the distances of each marker point relative to the first camera frame are 10, 20, 30, 40, and 50. The second position information includes the distances of three marker points in the current frame marker point image relative to the first camera frame (second position information) of 29, 38, and 49. At this time, the first position information and the second position information are matched in numerical value proximity, and the second target matching relationship obtained is 29 matches 30, 38 matches 40, and 49 matches 50. It is known that the marker point with a distance of 29 in the current frame marker point image relative to the first camera frame is the same marker point as the marker point coded 3 in the first position information, the marker point with a distance of 38 in the current frame marker point image relative to the first camera frame is the same marker point as the marker point coded 4 in the first position information, and the marker point with a distance of 49 in the current frame marker point image relative to the first camera frame is the same marker point as the marker point coded 5 in the first position information.

[0088] It can be seen that in this example, when the actual effective marking information includes scale factor information, image coordinate information and the first mileage information, two least squares optimizations are performed when obtaining the first position information, and the first position information is used as a reference for global matching. Therefore, when the matching relationship of the landmark points is obtained by matching the position information, it is beneficial to improve the matching accuracy.

[0089] In a possible example, after acquiring the mark information of the mark point, the method may include the following steps: if it is determined that the mark information includes the scale factor information, the image coordinate information and the coding mark information, determining a first target matching relationship according to the scale factor information and the image coordinate information; determining the image quality information of each mark point; screening out the target mark points whose image quality information meets the preset image conditions; determining the first absolute coding information of the target mark point according to the coding mark information; and determining the second absolute coding information of the mark point in the mark point image of the current frame according to the first target matching relationship and the first absolute coding information.

[0090] The image quality information includes image contrast, gradient and other indicators for judging image quality, which are not limited here. The preset image condition may be that the image contrast of the marker point is greater than or equal to a fourth preset threshold and the gradient is greater than or equal to a fifth preset threshold. The fourth preset threshold and the fifth preset threshold may be manually set or set by the system by default, and are not limited here.

[0091] Among them, the absolute coding information is the number of the mark point set in advance. Different mark points may correspond to different coding marks. Therefore, a correspondence between the coding mark and the absolute code can be established. After the coding mark of the mark point is identified, the absolute code of the mark point can be obtained accordingly.

[0092] Wherein, according to the first target matching relationship and the absolute coding information of the target marker point, the temporal coding of the current frame marker point image is replaced with the absolute coding. For example, the current frame marker point image includes four marker points, and the four marker points are defined as the first marker point, the second marker point, the third marker point and the fourth marker point from top to bottom. The previous frame marker point image includes marker points 1, 2, 3, and 4. The first target matching relationship is that the first marker point in the current frame marker point image matches the marker point coded 2 in the previous frame marker point image, and the second marker point in the current frame marker point image matches the marker point coded 3 in the previous frame marker point image. Matching, the third marker point in the marker point image of the current frame matches the marker point coded 4 in the marker point image of the previous frame. The first absolute coding information recognizes that the absolute codes of the first marker point and the fourth marker point are 77 and 69. The pre-stored absolute coding information table shows that the marker point codes are 77, 11, 18, and 22, respectively. The pre-stored absolute coding information table can be updated to 77, 11, 18, 22, and 69. At the same time, the marker points of the marker point image of the current frame are determined to be 2, 3, 4, and 5 in time sequence. At the same time, the time sequence codes can be replaced with 11, 18, 22, and 69 to obtain the second absolute coding information.

[0093] It can be seen that in this example, when the actual valid landmark information includes the scale factor, image coordinate information and coding mark information, while matching the timing coding, the image quality judgment is introduced to replace the timing coding with the absolute coding information, which is beneficial to improve the accuracy of landmark point matching.

[0094] In a possible example, after acquiring the mark information of the mark point, the method may include the following steps: if it is determined that the mark information includes the scale factor information, the image coordinate information, the first mileage information and the coding mark information, then determining a second target matching relationship according to the scale factor, the image coordinate information and the first mileage information; and determining second absolute coding information according to the scale factor information, the image coordinate information and the coding mark information.

[0095] Among them, when the valid and usable mark information includes scale factor information, image coordinate information, first mileage information and coding mark information, the above two mark point matching methods can be used at the same time to determine the second target matching relationship and the second absolute coding information respectively. If there is only a time sequence coding matching method, the mark point interval may be incorrect due to mismatching. If there is only a coding matching method, the correct coding may not be detected due to the poor clarity of the mark point. In addition, the mark point position information and the second absolute coding information can be compared, and damaged or missing mark points are abnormal mark points.

[0096] For example, if in a measurement, the second absolute coding information is 22 and 8, and the original absolute coding sequence should be 22, 69 and 8, and according to the analysis of the marker point position information, it is found that the distance interval between marker point 22 and marker point 8 is much larger than the pre-set distance interval, then the marker point with absolute code 69 is considered to be damaged or missing.

[0097] It can be seen that in this example, the landmark information includes scale factor information, image coordinate information, first mileage information and coding mark information, and the second target matching relationship and the second absolute coding information can be determined in parallel, which is beneficial to improving the accuracy of landmark point matching.

[0098] In a possible example, before acquiring the marker information of the marker point, the method may include the following steps: performing the following operations on each first marker point to obtain a first scale factor of each first marker point: determining a physical scale value between a first sub-marker and a second sub-marker of a first marker point currently being processed; determining a pixel scale value between the first sub-marker and the second sub-marker of the first marker point currently being processed imaged in the marker point image of the current frame; and determining the first scale factor of the first marker point currently being processed based on the physical scale value and the pixel scale value.

[0099] See also Figure 5 , Figure 5 is a schematic diagram of an object plane, an image plane and an optical center provided in an embodiment of the present application, Figure 5 The optical center C, the optical axis, the object plane and the image plane are shown in FIG. The physical scale of the marker point in the object plane is d, which can be the physical scale between the first sub-marker and the second sub-marker. For example, the distance between the centroid of the first sub-marker and the centroid of the second sub-marker is d, and the pixel scale of the marker point imaged in the image plane is , may be the pixel scale between the first sub-marker and the second sub-marker imaged in the image plane. For example, the distance between the centroid of the first sub-marker and the centroid of the second sub-marker imaged in the image plane is , then the scale factor of the landmark point is Each time the camera observes the object to be measured, the aforementioned method can be used to calibrate the scale factor, which means that each time the camera makes an observation, the scale factor calibration is completed once, that is, the real-time calibration of the scale factor is realized.

[0100] The centroid scale can replace the center point scale for the following reasons: Assume that the shape of a component in the landmark point is a circle with a radius of r. The origin of the world coordinate system is established at the center of the circle, and the plane where the circle is located is set as the world coordinate system W-XY plane, such as Figure 6 As shown, Figure 6 is a schematic diagram of a marker point and a camera provided in an embodiment of the present application. Figure 6 The circular marker points are shown in the figure. The circular marker points become ellipses in the image. The equation of the quadratic curve is known to be , which can be written in matrix form as , where the quadratic curve coefficient matrix C is:

[0101] Then the equation of a circle with its center at the origin of the world coordinate system is:

[0102] Convert it into matrix form, the coefficient matrix C1 is:

[0103] Assume that the world coordinate system W-XYZ rotates around the Z axis first , and then rotate around the current X axis , and finally rotate around the current Y axis , the directions of the axes of the rotated coordinate system are consistent with the camera coordinate system The directions of the axes are parallel. The rotation matrix R is:

[0104] in They are:

[0105]

[0106]

[0107] Then by translation vector Translate the coordinate axis to coincide with the camera coordinate system. Assume that the equivalent focal lengths of the camera in the x and y directions are , The coordinates of the principal points in the x and y directions are , . Then the projection matrix M from the world coordinate system W-XYZ to the image coordinate system O-xy is:

[0108] Wherein, O is a 3×3 zero matrix.

[0109] Because the points on the circle exist in the same plane, the homography matrix H can be used to represent the transformation relationship between the points in the image coordinate system and the points in the world coordinate system: , where H is

[0110] in Indicates the distance from the world point to the optical center of the camera along the optical axis.

[0111] The formula of C1 becomes:

[0112] That is, the matrix form of the elliptic curve in the image can be used To express it. The coordinates (u, v) of the centroid of the ellipse are expressed as:

[0113] Will Substituting the elements in into the above centroid coordinate expression, we get and . Further simplify the formula. From the symmetry of the circle, we know that rotating around the Z axis of the world coordinate system will not change its image, so we can set The camera's principal point coordinates will change the pixel coordinates of the ellipse center in the image, but will not change the difference between the ellipse center and the circle center projection point. , The simplified pixel coordinates of the ellipse center are:

[0114]

[0115] In most camera measurement systems, the pixel coordinates of the ellipse centroid can be further simplified to:

[0116] The coordinates of the image point of the center of the circle are:

[0117]

[0118] It can be known that the distance between the centroid coordinates in the image and the center image point coordinates is Mainly

[0119] and

[0120] The two terms are defined as a coefficient of variation

[0121] This coefficient represents the degree of deviation between the coordinates of the centroid and the center image point in the image.

[0122] Exploring the coefficient of deviation through numerical simulation Deviation from the centroid coordinates and the center image point coordinates The default simulation parameters under the harsh non-vertical shooting conditions are shown in Table 1 below: Table 1

[0123] By controlling the variable method, The size of the corresponding deviation coefficient is calculated The distance between the centroid coordinates and the center image point coordinates , draw with and is the independent variable, For the curve of the dependent variable, see Figure 7 and Figure 8 , Figure 7 is a schematic diagram of a simulation curve provided in an embodiment of the present application, Figure 8 It is a schematic diagram of another simulation curve provided in an embodiment of the present application.

[0124] The simulation results show that as increase, After a sharp decline, it approaches zero, showing a negative power function law; increase, It gradually increases and the growth rate becomes larger and larger, showing a positive power function growth law. Under the default simulation conditions, when the distance from the camera to the measuring point in the optical axis direction reaches more than 4000mm, that is, when the deviation coefficient is within 12500, the deviation between the centroid coordinates and the center image point coordinates is It can be controlled at the sub-pixel level. Therefore, it can be considered that when the camera is far enough from the mark (similarly, the camera equivalent focal length and the radius of the mark point are small enough) and the deviation coefficient is small enough, it is advisable to use the centroid scale instead of the circle center scale to obtain the scale factor.

[0125] It can be seen that in this example, the scale factor can be calibrated by the centroid scale of the landmark point, which is conducive to improving the accuracy of the scale factor calibration.

[0126] See also Fig. 9 , Fig. 9 is a structural schematic diagram of an electronic device provided in an embodiment of the present application, which is applied to a processing device in an image measurement system, wherein the image measurement system comprises the processing device, a camera platform, a marker point, and a camera arranged on the camera platform, wherein the processing device is connected to the camera, and the marker point comprises a first sub-marker and a second sub-marker, wherein the first sub-marker and the second sub-marker have a scale feature and a direction feature; Fig. 9 As shown, the electronic device includes a processor, a memory, a communication interface, and one or more programs, wherein the one or more programs are stored in the memory, and the one or more programs are configured to execute the following steps by the processor: Acquire marker information of the marker point, wherein the marker information includes scale factor information and / or image coordinate information and / or coding mark information of the marker point in multiple frames of marker point images and / or first mileage information of the camera platform; If it is determined that the mark information includes the scale factor information and the image coordinate information, determining a first matching relationship between a first scale factor of a first mark point in a mark point image of a current frame and a second scale factor of a second mark point in a mark point image of a previous frame according to the scale factor information; Determine a first target matching relationship between a first marker point in the marker point image of the current frame and a second marker point in the marker point image of the previous frame according to the image coordinate information and the first matching relationship; According to the first target matching relationship, the temporal encoding of the first marker point in the marker point image of the current frame is determined.

[0127] It can be seen that in the embodiment of the present application, the electronic device can first obtain the marker information of the marker point, the marker information includes the scale factor information and / or image coordinate information and / or the coding mark information and / or the first mileage information of the camera platform of the marker point in the multiple frames of the marker point image, and then if it is determined that the marker information includes the scale factor information and the image coordinate information, then according to the scale factor, determine the first matching relationship between the first scale factor of the first marker point in the current frame of the marker point image and the second scale factor of the second marker point in the previous frame of the marker point image, and then according to the image coordinate information and the first matching relationship, determine the first target matching relationship between the first marker point in the current frame of the marker point image and the second marker point in the previous frame of the marker point image, and finally, according to the first target matching relationship, determine the temporal encoding of the first marker point in the current frame of the marker point image. It can be achieved that when measuring on a mobile camera platform or measuring a marker point with a large displacement, the scale factor of the marker point in the image is matched, and then the matching of the marker point is achieved, and the matching relationship of the scale factor is verified according to the motion constraint through the image coordinate information, which is conducive to improving the accuracy of image measurement.

[0128] In a possible example, in determining, according to the scale factor information, a first matching relationship between a first scale factor of a first marker point in a marker point image of a current frame and a second scale factor of a second marker point in a marker point image of a previous frame, the program includes instructions for performing the following steps: Perform the following operation on the first scale factor of each first marker point in the marker point image of the current frame to obtain the first matching relationship: Determine the difference between the first scale factor currently being processed and each second scale factor to obtain a plurality of difference values; determine the absolute values ​​of the plurality of difference values ​​to obtain a plurality of target values; Determine a target second proportional factor corresponding to the smallest value among the multiple target values; If it is determined that the target value corresponding to the target second scale factor is less than the first preset threshold, it is determined that the currently processed first scale factor has a matching relationship with the target second scale factor.

[0129] In a possible example, in determining the first target matching relationship between the first marker point in the marker point image of the current frame and the second marker point in the marker point image of the previous frame according to the image coordinate information and the first matching relationship, the program includes instructions for performing the following steps: Perform the following operation on each set of mutually matching first proportional factors and second proportional factors in the first matching relationship to determine a first target matching relationship between the first marker point and the second marker point: Determine, according to the image coordinate information, a conversion relationship between a first marker point corresponding to a first scale factor currently being processed and a second marker point corresponding to a second scale factor currently being processed; According to the conversion relationship, construct a target equation group; Solving the target equation group to obtain a rotation matrix; According to the rotation matrix, a reprojection error is estimated to obtain a target reprojection error; If it is determined that the target reprojection error is less than or equal to the sixth preset threshold, it is determined that a first marker point corresponding to the currently processed first scale factor and a second marker point corresponding to the currently processed second scale factor have a matching relationship.

[0130] In a possible example, after acquiring the marker information of the marker point, the program includes instructions for executing the following steps: If it is determined that the marker information includes the scale factor information, the image coordinate information and the first mileage information, predicting the mileage prediction information and the scale factor prediction information of the marker point image of the current frame according to the second scale factor of the second marker point in the marker point image of the previous frame, the first mileage information and a preset prediction model; Determine, according to the scale factor prediction information and the first scale factor of the marker point image of the current frame, a second matching relationship between a third scale factor in the scale factor prediction information and the first scale factor of the marker point image of the current frame; Correcting the preset prediction model according to the second matching relationship to obtain a corrected preset prediction model; Determine, according to the scale factor information, a first distance from each marker point in the multiple-frame marker point images to the optical center of the camera; Determining second mileage information according to the first distance; Determine first position information of the marker point in the multiple-frame marker point image according to the second mileage information and the scale factor information; Determine second position information of the first marker point according to the first mileage information and the scale factor information; The first position information and the second position information are matched to obtain a second target matching relationship of the marker point position.

[0131] In a possible example, after acquiring the marker information of the marker point, the program further includes instructions for executing the following steps: If it is determined that the mark information includes the scale factor information, the image coordinate information and the encoding mark information, determining a first target matching relationship according to the scale factor information and the image coordinate information; Determine image quality information for each landmark point; Filter out target landmark points whose image quality information meets preset image conditions; Determining first absolute coding information of the target marker point according to the coding mark information; According to the first target matching relationship and the first absolute coding information, second absolute coding information of the marker point in the marker point image of the current frame is determined.

[0132] In a possible example, after acquiring the marker information of the marker point, the program further includes instructions for executing the following steps: If it is determined that the mark information includes the scale factor information, the image coordinate information, the first mileage information and the coded mark information, determining a second target matching relationship according to the scale factor, the image coordinate information and the first mileage information; The second absolute coding information is determined according to the scale factor information, the image coordinate information and the coding mark information.

[0133] In a possible example, before acquiring the marker information of the marker point, the program further includes instructions for executing the following steps: The following operation is performed on each first marker point to obtain the first scale factor of each first marker point: Determine a physical dimension value between a first sub-marker and a second sub-marker of a first marker point currently being processed; Determine a pixel scale value between a first sub-marker and a second sub-marker of the first marker point currently being processed imaged in the marker point image of the current frame; A first scale factor of the first marker point currently being processed is determined according to the physical scale value and the pixel scale value.

[0134] In a possible example, after determining the temporal encoding of the first marker point in the marker point image of the current frame according to the first target matching relationship, the program further includes instructions for executing the following steps: If it is detected that there is a first target marker point that does not match the second marker point in the marker point image of the current frame, determine a fourth scale factor with the smallest scale factor value and a fifth scale factor with the largest scale factor value in the marker point image of the previous frame; Determine a difference between a first scale factor corresponding to the first target landmark point and the fourth scale factor to obtain a first target difference; Determine a difference between a first scale factor corresponding to the first target landmark and the fifth scale factor to obtain a second target difference; If it is determined that the first target difference is less than the second preset threshold or the second target difference is greater than the third preset threshold, the encoding of the first target marker point is determined based on the order of appearance of the first target marker points and the existing encoding information of the first marker point in the current frame marker point image.

[0135] The above mainly introduces the scheme of the embodiment of the present application from the perspective of the execution process on the method side. It is understandable that, in order to realize the above functions, the electronic device includes a hardware structure and / or software module corresponding to the execution of each function. It should be easily appreciated by those skilled in the art that, in combination with the units and algorithm steps of each example described in the embodiments provided herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present application.

[0136] The embodiment of the present application can divide the electronic device into functional units according to the above method example. For example, each functional unit can be divided according to each function, or two or more functions can be integrated into one processing unit. The above integrated unit can be implemented in the form of hardware or in the form of software functional units. It should be noted that the division of units in the embodiment of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation.

[0137] In the case of dividing each functional module into corresponding functional modules, Fig.10 is a functional unit block diagram of a landmark point matching device provided in an embodiment of the present application, such as Fig.10 As shown, the device includes an acquisition unit 1001 and a determination unit 1002; wherein, The acquisition unit 1001 is used to acquire the mark information of the mark point, where the mark information includes the scale factor information and / or the image coordinate information and / or the coding mark information and / or the first mileage information of the camera platform of the mark point in the multiple frames of the mark point image; The determining unit 1002 is configured to determine, if it is determined that the marker information includes the scale factor information and the image coordinate information, a first matching relationship between a first scale factor of a first marker point in a marker point image of a current frame and a second scale factor of a second marker point in a marker point image of a previous frame according to the scale factor information; The determining unit 1002 is further configured to determine a first target matching relationship between a first marker point in the marker point image of the current frame and a second marker point in the marker point image of the previous frame according to the image coordinate information and the first matching relationship; The determining unit 1002 is further configured to determine the temporal encoding of the first marker point in the marker point image of the current frame according to the first target matching relationship.

[0138] It can be seen that in the embodiment of the present application, the marker point matching device can first obtain the marker information of the marker point, the marker information includes the scale factor information and / or image coordinate information and / or the coding mark information and / or the first mileage information of the camera platform of the marker point in the multiple frames of the marker point image, and then if it is determined that the marker information includes the scale factor information and the image coordinate information, then according to the scale factor, determine the first matching relationship between the first scale factor of the first marker point in the current frame of the marker point image and the second scale factor of the second marker point in the previous frame of the marker point image, and then according to the image coordinate information and the first matching relationship, determine the first target matching relationship between the first marker point in the current frame of the marker point image and the second marker point in the previous frame of the marker point image, and finally, according to the first target matching relationship, determine the temporal encoding of the first marker point in the current frame of the marker point image. It can be achieved that when measuring on a mobile camera platform or measuring a marker point with a large displacement, the scale factor of the marker point in the image is matched, and then the matching of the marker point is achieved, and the matching relationship of the scale factor is verified according to the motion constraint through the image coordinate information, which is conducive to improving the accuracy of image measurement.

[0139] In a possible example, in determining, according to the scale factor information, a first matching relationship between a first scale factor of a first marker point in a marker point image of a current frame and a second scale factor of a second marker point in a marker point image of a previous frame, the determining unit 1002 is specifically configured to: Perform the following operation on the first scale factor of each first marker point in the marker point image of the current frame to obtain the first matching relationship: Determine the difference between the first scale factor currently being processed and each second scale factor to obtain a plurality of difference values; determine the absolute values ​​of the plurality of difference values ​​to obtain a plurality of target values; Determine a target second proportional factor corresponding to the smallest value among the multiple target values; If it is determined that the target value corresponding to the target second scale factor is less than the first preset threshold, it is determined that the currently processed first scale factor has a matching relationship with the target second scale factor.

[0140] In a possible example, in determining the first target matching relationship between the first marker point in the marker point image of the current frame and the second marker point in the marker point image of the previous frame according to the image coordinate information and the first matching relationship, the determining unit 1002 is specifically configured to: Perform the following operation on each set of mutually matching first proportional factors and second proportional factors in the first matching relationship to determine a first target matching relationship between the first marker point and the second marker point: Determine, according to the image coordinate information, a conversion relationship between a first marker point corresponding to a first scale factor currently being processed and a second marker point corresponding to a second scale factor currently being processed; According to the conversion relationship, construct a target equation group; Solving the target equation group to obtain a rotation matrix; According to the rotation matrix, a reprojection error is estimated to obtain a target reprojection error; If it is determined that the target reprojection error is less than or equal to the sixth preset threshold, it is determined that a first marker point corresponding to the currently processed first scale factor and a second marker point corresponding to the currently processed second scale factor have a matching relationship.

[0141] In a possible example, after acquiring the marker information of the marker point, the determining unit 1002 is specifically configured to: If it is determined that the marker information includes the scale factor information, the image coordinate information and the first mileage information, predicting the mileage prediction information and the scale factor prediction information of the marker point image of the current frame according to the second scale factor of the second marker point in the marker point image of the previous frame, the first mileage information and a preset prediction model; Determine, according to the scale factor prediction information and the first scale factor of the marker point image of the current frame, a second matching relationship between a third scale factor in the scale factor prediction information and the first scale factor of the marker point image of the current frame; Correcting the preset prediction model according to the second matching relationship to obtain a corrected preset prediction model; Determine, according to the scale factor information, a first distance from each marker point in the multiple-frame marker point images to the optical center of the camera; Determining second mileage information according to the first distance; Determine first position information of the marker point in the multiple-frame marker point image according to the second mileage information and the scale factor information; Determine second position information of the first marker point according to the first mileage information and the scale factor information; The first position information and the second position information are matched to obtain a second target matching relationship of the marker point position.

[0142] In a possible example, after acquiring the marker information of the marker point, the determining unit 1002 is further specifically configured to: If it is determined that the mark information includes the scale factor information, the image coordinate information and the encoding mark information, determining a first target matching relationship according to the scale factor information and the image coordinate information; Determine image quality information for each landmark point; Filter out target landmark points whose image quality information meets preset image conditions; Determining first absolute coding information of the target marker point according to the coding mark information; According to the first target matching relationship and the first absolute coding information, second absolute coding information of the marker point in the marker point image of the current frame is determined.

[0143] In a possible example, after acquiring the marker information of the marker point, the determining unit 1002 is further specifically configured to: If it is determined that the mark information includes the scale factor information, the image coordinate information, the first mileage information and the coded mark information, determining a second target matching relationship according to the scale factor, the image coordinate information and the first mileage information; The second absolute coding information is determined according to the scale factor information, the image coordinate information and the coding mark information.

[0144] In a possible example, before acquiring the marker information of the marker point, the determining unit 1002 is further specifically configured to: The following operation is performed on each first marker point to obtain the first scale factor of each first marker point: Determine a physical dimension value between a first sub-marker and a second sub-marker of a first marker point currently being processed; Determine a pixel scale value between a first sub-marker and a second sub-marker of the first marker point currently being processed imaged in the marker point image of the current frame; A first scale factor of the first marker point currently being processed is determined according to the physical scale value and the pixel scale value.

[0145] In a possible example, after determining the temporal encoding of the first marker point in the marker point image of the current frame according to the first target matching relationship, the determining unit 1002 is further specifically configured to: If it is detected that there is a first target marker point that does not match the second marker point in the marker point image of the current frame, determine a fourth scale factor with the smallest scale factor value and a fifth scale factor with the largest scale factor value in the marker point image of the previous frame; Determine a difference between a first scale factor corresponding to the first target landmark point and the fourth scale factor to obtain a first target difference; Determine a difference between a first scale factor corresponding to the first target landmark and the fifth scale factor to obtain a second target difference; If it is determined that the first target difference is less than the second preset threshold or the second target difference is greater than the third preset threshold, the encoding of the first target marker point is determined based on the order of appearance of the first target marker points and the existing encoding information of the first marker point in the current frame marker point image.

[0146] It should be noted that all relevant contents of each step involved in the above method embodiment can be referred to the functional description of the corresponding functional module and will not be repeated here.

[0147] The electronic device provided in this embodiment is used to execute the above-mentioned landmark point matching method, and thus can achieve the same effect as the above-mentioned implementation method.

[0148] In the case of an integrated unit, the electronic device may include a processing module, a storage module and a communication module. The processing module may be used to control and manage the actions of the electronic device, for example, it may be used to support the electronic device to execute the steps performed by the above-mentioned functional unit. The storage module may be used to support the electronic device to execute stored program codes and data, etc. The communication module may be used to support the communication between the electronic device and other devices.

[0149] Among them, the processing module can be a processor or a controller. It can implement or execute various exemplary logic boxes, modules and circuits described in conjunction with the disclosure of this application. The processor can also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of digital signal processing (DSP) and a microprocessor, etc. The storage module can be a memory. The communication module can specifically be a device that interacts with other electronic devices, such as a radio frequency circuit, a Bluetooth chip, a Wi-Fi chip, etc.

[0150] An embodiment of the present application also provides a computer storage medium, wherein the computer storage medium stores a computer program for electronic data exchange, wherein the computer program enables a computer to execute part or all of the steps of any method described in the above method embodiments, and the above computer includes an electronic device.

[0151] The present application also provides a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to cause a computer to execute some or all of the steps of any method described in the method embodiment. The computer program product may be a software installation package, and the computer includes a control platform.

[0152] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0153] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0154] In the several embodiments provided in the present application, it should be understood that the disclosed device can be implemented in other ways. For example, the device embodiments described above are only schematic, such as the division of the above-mentioned units, which is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection of devices or units can be electrical or other forms.

[0155] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0156] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0157] If the above-mentioned integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a memory, including a number of instructions to enable a computer device (which can be a personal computer, server or network device, etc.) to execute all or part of the steps of the above-mentioned methods in each embodiment of the present application. The aforementioned memory includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, disk or optical disk and other media that can store program codes.

[0158] A person of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable memory, which can include: a flash drive, a read-only memory, a random access memory, a magnetic disk or an optical disk, etc.

[0159] The embodiments of the present application are introduced in detail above. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method and core idea of ​​the present application. At the same time, for general technical personnel in this field, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A landmark point matching method, characterized in that: A processing device applied to an image measurement system, the image measurement system comprising the processing device, a camera platform, a marker point and a camera arranged on the camera platform, the processing device being connected to the camera, the marker point comprising a first sub-marker and a second sub-marker, the first sub-marker and the second sub-marker having a scale feature and a direction feature; the method comprising: Acquire marker information of the marker point, wherein the marker information includes scale factor information and / or image coordinate information and / or coding mark information of the marker point in multiple frames of marker point images and / or first mileage information of the camera platform; If it is determined that the mark information includes the scale factor information and the image coordinate information, determining a first matching relationship between a first scale factor of a first mark point in a mark point image of a current frame and a second scale factor of a second mark point in a mark point image of a previous frame according to the scale factor information; Determine a first target matching relationship between a first marker point in the marker point image of the current frame and a second marker point in the marker point image of the previous frame according to the image coordinate information and the first matching relationship; According to the first target matching relationship, the temporal encoding of the first marker point in the marker point image of the current frame is determined.

2. The method according to claim 1, characterized in that The determining, according to the scale factor information, a first matching relationship between a first scale factor of a first marker point in a current frame marker point image and a second scale factor of a second marker point in a previous frame marker point image comprises: Perform the following operation on the first scale factor of each first marker point in the marker point image of the current frame to obtain the first matching relationship: Determine a difference between the first scale factor currently being processed and each second scale factor to obtain a plurality of difference values; Determine the absolute values ​​of the plurality of differences to obtain a plurality of target values; Determine a target second proportional factor corresponding to the smallest value among the multiple target values; If it is determined that the target value corresponding to the target second scale factor is less than the first preset threshold, it is determined that the currently processed first scale factor has a matching relationship with the target second scale factor.

3. The method according to claim 1, characterized in that The determining, according to the image coordinate information and the first matching relationship, a first target matching relationship between a first marker point in the marker point image of the current frame and a second marker point in the marker point image of the previous frame includes: Perform the following operation on each set of mutually matching first proportional factors and second proportional factors in the first matching relationship to determine a first target matching relationship between the first marker point and the second marker point: Determine, according to the image coordinate information, a conversion relationship between a first marker point corresponding to a first scale factor currently being processed and a second marker point corresponding to a second scale factor currently being processed; According to the conversion relationship, construct a target equation group; Solving the target equation group to obtain a rotation matrix; According to the rotation matrix, a reprojection error is estimated to obtain a target reprojection error; If it is determined that the target reprojection error is less than or equal to the sixth preset threshold, it is determined that a first marker point corresponding to the currently processed first scale factor and a second marker point corresponding to the currently processed second scale factor have a matching relationship.

4. The method according to claim 1, characterized in that After acquiring the mark information of the mark point, the method further includes: If it is determined that the marker information includes the scale factor information, the image coordinate information and the first mileage information, predicting the mileage prediction information and the scale factor prediction information of the marker point image of the current frame according to the second scale factor of the second marker point in the marker point image of the previous frame, the first mileage information and a preset prediction model; Determine, according to the scale factor prediction information and the first scale factor of the marker point image of the current frame, a second matching relationship between a third scale factor in the scale factor prediction information and the first scale factor of the marker point image of the current frame; Correcting the preset prediction model according to the second matching relationship to obtain a corrected preset prediction model; Determine, according to the scale factor information, a first distance from each marker point in the multiple-frame marker point images to the optical center of the camera; Determining second mileage information according to the first distance; Determine first position information of the marker point in the multiple-frame marker point image according to the second mileage information and the scale factor information; Determine second position information of the first marker point according to the first mileage information and the scale factor information; The first position information and the second position information are matched to obtain a second target matching relationship of the marker point position.

5. The method according to claim 1, characterized in that After acquiring the mark information of the mark point, the method further includes: If it is determined that the mark information includes the scale factor information, the image coordinate information and the encoding mark information, determining a first target matching relationship according to the scale factor information and the image coordinate information; Determine image quality information for each landmark point; Filter out target landmark points whose image quality information meets preset image conditions; Determining first absolute coding information of the target marker point according to the coding mark information; According to the first target matching relationship and the first absolute coding information, second absolute coding information of the marker point in the marker point image of the current frame is determined.

6. The method according to claim 1, characterized in that After acquiring the mark information of the mark point, the method further includes: If it is determined that the mark information includes the scale factor information, the image coordinate information, the first mileage information and the coded mark information, determining a second target matching relationship according to the scale factor, the image coordinate information and the first mileage information; The second absolute coding information is determined according to the scale factor information, the image coordinate information and the coding mark information.

7. The method according to claim 1, characterized in that Before acquiring the mark information of the mark point, the method further includes: The following operation is performed on each first marker point to obtain the first scale factor of each first marker point: Determine a physical dimension value between a first sub-marker and a second sub-marker of a first marker point currently being processed; Determine a pixel scale value between a first sub-marker and a second sub-marker of the first marker point currently being processed imaged in the marker point image of the current frame; A first scale factor of the first marker point currently being processed is determined according to the physical scale value and the pixel scale value.

8. The method according to claim 1, characterized in that After determining the temporal encoding of the first marker point in the marker point image of the current frame according to the first target matching relationship, the method further includes: If it is detected that there is a first target marker point that does not match the second marker point in the marker point image of the current frame, determine a fourth scale factor with the smallest scale factor value and a fifth scale factor with the largest scale factor value in the marker point image of the previous frame; Determine a difference between a first scale factor corresponding to the first target landmark point and the fourth scale factor to obtain a first target difference; Determine a difference between a first scale factor corresponding to the first target landmark point and the fifth scale factor to obtain a second target difference; If it is determined that the first target difference is less than the second preset threshold or the second target difference is greater than the third preset threshold, the encoding of the first target marker point is determined based on the order of appearance of the first target marker points and the existing encoding information of the first marker point in the current frame marker point image.

9. A landmark point matching device, characterized in that: A processing device applied to an image measurement system, the image measurement system comprising the processing device, a camera platform, a marker point and a camera arranged on the camera platform, the processing device is connected to the camera, the marker point comprises a first sub-marker and a second sub-marker, the first sub-marker and the second sub-marker have a scale feature and a direction feature; the marker point matching device comprises an acquisition unit and a determination unit, wherein, The acquisition unit is used to acquire the mark information of the mark point, wherein the mark information includes the scale factor information and / or the image coordinate information and / or the coding mark information and / or the first mileage information of the camera platform of the mark point in the multiple frames of the mark point image; The determining unit is configured to determine, if it is determined that the marker information includes the scale factor information and the image coordinate information, a first matching relationship between a first scale factor of a first marker point in a marker point image of a current frame and a second scale factor of a second marker point in a marker point image of a previous frame according to the scale factor information; The determining unit is further configured to determine a first target matching relationship between a first marker point in the marker point image of the current frame and a second marker point in the marker point image of the previous frame according to the image coordinate information and the first matching relationship; The determining unit is further configured to determine the temporal encoding of the first marker point in the marker point image of the current frame according to the first target matching relationship.

10. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the memory is used to store one or more programs and is configured to be executed by the processor, wherein the program comprises instructions for executing the steps in the method according to any one of claims 1 to 7.

Images