Picture authenticity checking method and device, and storage medium

Abstract

The application discloses a picture authenticity inspection method and device and a storage medium, and relates to the technical field of data processing. The method comprises the following steps: acquiring a target video frame corresponding to a time label of a to-be-inspected picture from an inspection room video according to the time label; determining feature points of an inspected article in the target video frame and an inspection article in the to-be-inspected picture, determining a first pose parameter of a camera according to the feature points and camera internal parameters; determining a preset feature point of the camera in the target video frame, determining a second pose parameter of the camera according to the preset feature point and camera parameters, and predicting a shooting picture according to the second pose parameter; determining a parameter error of the first pose parameter and the second pose parameter, and an image error of the to-be-inspected picture and the shooting picture, and determining that the to-be-inspected picture passes the inspection when the parameter error and the image error are both less than a preset error threshold, so that the authenticity of the picture is inspected.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, in particular to a picture authenticity verification method, device and storage medium. BACKGROUND

[0002] In the cosmetic inspection scene, the real image data attached to the inspection report is one of the core evidences to verify the compliance of the inspection process. The related image verification method is usually to verify the picture by embedding invisible watermark or using MD5 (Message-Digest Algorithm 5) / SHA (Secure Hash Algorithm) algorithm to generate digital fingerprints. However, the above-mentioned means essentially belongs to the file integrity verification tool, and its verification logic is based on the assumption that the file has not been modified, and it cannot verify the tampering of the image content level. For example, by pre-shooting or storing the tampered "qualified" historical image, the historical image is directly used to replace the real-time shooting of the inspected object image in the inspection link, or the inspected object is replaced by other qualified objects for shooting. Therefore, the above-mentioned image verification method cannot verify the authenticity of the data.

[0003] The above content is only used to assist in understanding the technical solutions of the present application, and does not represent the acknowledgement of the above content as prior art. SUMMARY

[0004] The main purpose of the present application is to provide a picture authenticity verification method, device and storage medium, which aims to solve the technical problem that the current image verification method cannot verify the authenticity of the data.

[0005] To achieve the above purpose, the present application provides a picture authenticity verification method, which comprises:

[0006] According to the time label of the picture to be inspected, a target video frame corresponding to the time label is obtained from the inspection room video;

[0007] Determine the feature points of the inspected object in the target video frame and the inspection object in the picture to be inspected, and determine the first pose parameter of the camera according to the feature points and the camera intrinsic parameter;

[0008] Determine the preset feature points of the camera in the target video frame, determine the second pose parameter of the camera according to the preset feature points and the camera parameter, and predict the shooting picture according to the second pose parameter;

[0009] Determine the parameter error of the first pose parameter and the second pose parameter, and the image error of the picture to be inspected and the shooting picture, and determine that the picture to be inspected passes the inspection when the parameter error and the image error are both less than a preset error threshold.

[0010] In an embodiment, the first pose parameter comprises a first rotation matrix and a first translation vector, and the step of determining the feature points of the inspected item in the target video frame and the inspection item in the picture to be inspected according to the feature points and the camera intrinsic parameter comprises:

[0011] identifying a texture feature of the inspection item to obtain a two-dimensional feature point of the inspection item;

[0012] matching the two-dimensional feature point with a three-dimensional model of the inspected item to obtain a three-dimensional feature point corresponding to the two-dimensional feature point;

[0013] determining the first rotation matrix and the first translation vector of the camera according to a projection relationship between the two-dimensional feature point and the three-dimensional feature point and the camera intrinsic parameter.

[0014] In an embodiment, the step of determining the first rotation matrix and the first translation vector of the camera according to the projection relationship between the two-dimensional feature point and the three-dimensional feature point and the camera intrinsic parameter further comprises:

[0015] inputting the two-dimensional feature point, the three-dimensional feature point and the camera intrinsic parameter into a preset EPnP model, representing the three-dimensional feature point as a weighted sum of control points through the EPnP model, and determining coordinates of the control points in a camera coordinate system according to the projection relationship and the camera intrinsic parameter;

[0016] determining the first rotation matrix and the first translation vector of the camera according to the coordinates of the control points in the camera coordinate system and coordinates of the control points in a world coordinate system.

[0017] In an embodiment, the first pose parameter comprises a first shooting distance, and the step of determining the feature points of the inspected item in the target video frame and the inspection item in the picture to be inspected according to the feature points and the camera intrinsic parameter comprises:

[0018] determining a pixel size of the inspection item according to the two-dimensional feature point of the inspection item;

[0019] determining an actual size of the inspected item according to the three-dimensional feature point of the inspected item;

[0020] determining the first shooting distance of the camera according to the pixel size of the inspection item, the actual size of the inspected item and the camera intrinsic parameter.

[0021] In an embodiment, the second pose parameter comprises a second rotation matrix and a second translation vector, and the step of determining the preset feature point of the camera in the target video frame and determining the second pose parameter of the camera according to the preset feature point and the camera parameter comprises:

[0022] determining the pixel coordinates of the preset feature point of the camera in each target video frame;

[0023] projecting the preset feature point to a world coordinate system according to the pixel coordinates and the camera parameter corresponding to each target video frame to obtain a first projection equation of each preset feature point;

[0024] determining the solution of the first projection equation according to the least square method to obtain the coordinates of each preset feature point in the world coordinate system;

[0025] determining the second rotation matrix and the second translation vector of the camera according to the coordinates of each preset feature point in the world coordinate system and the coordinates of each preset feature point in the camera coordinate system.

[0026] In an embodiment, the camera parameter comprises a camera intrinsic parameter and a camera extrinsic parameter, and the step of projecting the preset feature point to a world coordinate system according to the pixel coordinates and the camera parameter corresponding to each target video frame to obtain a first projection equation of each preset feature point comprises:

[0027] projecting the preset feature point to a camera coordinate system according to the pixel coordinates and the camera intrinsic parameter corresponding to each target video frame to obtain a second projection equation of each preset feature point;

[0028] projecting the preset feature point to a world coordinate system according to the second projection equation of each preset feature point and the camera extrinsic parameter corresponding to each target video frame to obtain a first projection equation of each preset feature point.

[0029] In an embodiment, the second pose parameter comprises a second shooting distance, and after the step of determining the rotation matrix and the translation vector of the camera according to the coordinates of each preset feature point in the world coordinate system and the coordinates of each preset feature point in the camera coordinate system, the method further comprises:

[0030] determining the coordinates of the camera in the world coordinate system according to the coordinates of the preset feature point in the world coordinate system;

[0031] determining the second shooting distance according to the coordinates of the camera in the world coordinate system and the coordinates of the inspected object in the world coordinate system.

[0032] In an embodiment, after the step of acquiring the target video frame corresponding to the time label from the inspection room video according to the time label of the picture to be inspected, the method further comprises:

[0033] determining the similarity between the inspected item in the target video frame and the inspection item in the picture to be inspected;

[0034] when the similarity is greater than a preset similarity threshold, performing the step of determining the feature point of the inspected item in the target video frame and the inspection item in the picture to be inspected.

[0035] In addition, to achieve the above-mentioned purpose, the present application also provides a picture authenticity inspection device, which comprises a memory, a processor and a computer program stored on the memory and executable on the processor, and the computer program is configured to implement the steps of the picture authenticity inspection method as described above.

[0036] In addition, to achieve the above-mentioned purpose, the present application also provides a storage medium, which is a computer readable storage medium, and a computer program is stored on the storage medium, and the computer program is executed by a processor to implement the steps of the picture authenticity inspection method as described above.

[0037] The present application provides a picture authenticity inspection method, which acquires a target video frame corresponding to the time label from an inspection room video according to the time label of a picture to be inspected; determines the feature point of the inspected item in the target video frame and the inspection item in the picture to be inspected, determines the first pose parameter of the camera according to the feature point and the camera intrinsic parameter; determines the preset feature point of the camera in the target video frame, determines the second pose parameter of the camera according to the preset feature point and the camera parameter, and predicts a shooting picture according to the second pose parameter; determines the parameter error of the first pose parameter and the second pose parameter, and the image error of the picture to be inspected and the shooting picture, and determines that the picture to be inspected passes the inspection when the parameter error and the image error are both less than a preset error threshold.

[0038] The above-mentioned method inversely deduces the first pose parameter and the second pose parameter of the camera according to the picture to be inspected and the inspection room video respectively, and predicts a rendered shooting image according to the second pose parameter of the camera in the inspection room video. If the user temporarily replaces the inspected item with another qualified product, or uses a "qualified picture" to impersonate, the first pose parameter and the second pose parameter of the camera, and the picture to be inspected and the rendered shooting image will have a large difference, so as to be detected, and the picture authenticity inspection is realized. BRIEF DESCRIPTION OF DRAWINGS

[0039] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and serve to explain the principles of the present application together with the specification.

[0040] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the accompanying drawings needed to be used in the embodiments or prior art description will be briefly introduced as follows. Obviously, for those of ordinary skill in the art, the other drawings can also be obtained based on these drawings without any creative effort.

[0041] Figure 1 A flowchart provided by the picture authenticity verification method embodiment one of the present application;

[0042] Figure 2 A flowchart provided by the picture authenticity verification method embodiment two of the present application;

[0043] Figure 3 A flowchart provided by the picture authenticity verification method embodiment three of the present application;

[0044] Figure 4 A flowchart provided by the picture authenticity verification method embodiment four of the present application;

[0045] Figure 5 A device structure schematic diagram of the hardware running environment involved in the picture authenticity verification method in the embodiments of the present application.

[0046] The object implementation, functional features and advantages of the present application will be further described with reference to the embodiments and the accompanying drawings. DETAILED DESCRIPTION

[0047] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application, and are not used to limit the present application.

[0048] In order to better understand the technical solutions of the present application, the following will be described in detail in combination with the drawings in the specification and specific embodiments. It should be noted that all the actions of obtaining signals, information or data in the present application are carried out under the premise of complying with the corresponding data protection regulations and policies of the country where the device is located, and with the authorization given by the corresponding device owner.

[0049] In the cosmetic inspection scene, the physical image data attached to the inspection report is one of the core evidences to verify the compliance of the inspection process. The related image verification method is usually to verify the picture by embedding invisible watermark or using MD5 / SHA algorithm to generate digital fingerprint. However, the above-mentioned means essentially belongs to the file integrity verification tool, and its verification logic is based on the assumption that the file has not been modified, and it cannot verify the tampering of the image content level. For example, by pre-shooting or storing the tampered "qualified" historical image, the historical image is directly used to replace the real-time shooting of the inspected product image in the inspection link, or the inspected product is replaced by other qualified products for shooting. Therefore, the above-mentioned image verification method cannot verify the authenticity of the data.

[0050] In view of the above problems, the present application provides a picture authenticity inspection method, according to the time label of the picture to be inspected, the target video frame corresponding to the time label is obtained from the inspection room video; the feature points of the inspected product in the target video frame and the inspection product in the picture to be inspected are determined, and the first pose parameter of the camera is determined according to the feature points and the camera internal parameter; the preset feature points of the camera in the target video frame are determined, the second pose parameter of the camera is determined according to the preset feature points and the camera parameter, and the shooting picture is predicted according to the second pose parameter; the parameter error of the first pose parameter and the second pose parameter, and the image error of the picture to be inspected and the shooting picture are determined, and when the parameter error and the image error are less than the preset error threshold, it is determined that the picture to be inspected passes the inspection.

[0051] The above-mentioned method inversely deduces the first pose parameter and the second pose parameter of the camera according to the picture to be inspected and the inspection room video respectively, and renders the shooting image according to the second pose parameter of the camera in the inspection room video. If the user temporarily replaces the inspected product with another qualified product, or uses a "qualified picture" to impersonate, the first pose parameter and the second pose parameter of the camera, and the picture to be inspected and the rendered shooting image will have a large difference, so as to be detected, and the picture authenticity inspection is realized.

[0052] It should be noted that the execution subject of the present embodiment can be a computing service device with data processing, network communication and program running functions, such as a tablet computer, a personal computer, etc., or an electronic device, a picture inspection system, etc. capable of realizing the above-mentioned functions. The present embodiment and the following embodiments will be described below taking the picture inspection system as an example.

[0053] Based on this, the first embodiment of the present application provides a picture authenticity inspection method, which is described with reference to Figure 1 In the present embodiment, the picture authenticity inspection method comprises steps S10-S40:

[0054] Step S10, according to the time label of the picture to be inspected, the target video frame corresponding to the time label is obtained from the inspection room video.

[0055] It should be noted that the picture to be inspected refers to the product picture that needs to be inspected on the inspection report. Taking the inspection of cosmetics as an example, the picture to be inspected can be the picture of the appearance and texture of the cosmetics. The inspection room video is a global video stream recorded by a fixed camera in the inspection room in real time during the inspection process. Multiple fixed cameras are arranged at different positions in the inspection room to cover the operation area of the inspected object.

[0056] Exemplarily, when the camera shoots, it will automatically write the time stamp into the EXIF (Exchangeable Image File Format) metadata field of the picture, such as the DateTimeOriginal field. When the picture inspection system obtains the picture to be inspected, the shooting time stamp, i.e. the time label, can be determined from the above metadata field of the picture to be inspected. Then, according to the time label, the single frame image corresponding to the time label is obtained from the inspection room video recorded by each camera, and the above target video frame is obtained.

[0057] It can be understood that in the above steps, the picture to be inspected is anchored to the inspection process in the physical world through the time label, which can ensure that the picture to be inspected and the inspection room video strictly correspond in the time dimension. If a pre-stored qualified picture is used, its time label cannot match the actual operation time in the inspection room video, so it will be detected.

[0058] Step S20, determining the feature points of the inspected object in the target video frame and the inspection object in the picture to be inspected, and determining the first pose parameter of the camera according to the feature points and the camera intrinsic parameter.

[0059] The above-mentioned inspected object refers to the actual physical object recorded in the target video frame during the inspection process. The inspection object is the photographed object displayed in the picture to be inspected.

[0060] It can be understood that, since there are usually multiple cameras in the inspection room, each target video frame contains images of the inspected item from different angles. A three-dimensional model of the inspected item can be constructed from the images of the inspected item in each target video frame, and then the two-dimensional feature points of the inspected item in the picture to be inspected are matched with the three-dimensional model of the inspected item to determine the three-dimensional feature points corresponding to the two-dimensional feature points of the inspected item. Then, according to the two-dimensional feature points and the corresponding three-dimensional feature points, the first pose parameter of the camera is calculated in combination with the camera intrinsic parameter. The first pose parameter can include a first rotation matrix, a first translation vector and a first shooting distance. The camera intrinsic parameter is an inherent parameter describing the imaging characteristics of the camera itself, which does not change with the position of the camera or the shooting scene, and is used to project a three-dimensional space point to a two-dimensional image plane, including focal length, principal point coordinates and distortion parameters.

[0061] Alternatively, the product identification of the inspected item such as product labels on cosmetic products can also be recognized from each target video frame through a preset feature recognition model, and then a three-dimensional model of the inspected item pre-stored by the picture inspection system is loaded according to the product identification, and then the two-dimensional feature points of the inspected item in the picture to be inspected are matched with the three-dimensional model of the inspected item to determine the three-dimensional feature points corresponding to the two-dimensional feature points of the inspected item.

[0062] In step S30, a preset feature point of the camera in the target video frame is determined, a second pose parameter of the camera is determined according to the preset feature point and the camera parameter, and a shooting picture is predicted according to the second pose parameter.

[0063] Alternatively, the preset feature point on the camera is recognized from each target video frame through computer vision technology such as object detection, feature point matching or deep learning model.

[0064] It should be noted that the preset feature point is a non-coplanar point pre-marked on the camera, which can be a label or a specific position on the camera, such as a point on the edge of the lens. It can be understood that the coordinates of these preset feature points in the camera coordinate system are known, and their positions in the camera coordinate system are fixed and unchanged.

[0065] Since the cameras in the inspection room are fixed and their positions are known, the two-dimensional pixel coordinates of the detected preset feature points in different target video frames can be converted into three-dimensional coordinates in the world coordinate system by using multi-view geometry such as triangulation, and then the second pose parameter of the camera is determined according to the coordinates of the preset feature points in the world coordinate system and the coordinates in the camera coordinate system.

[0066] After determining the coordinates of the camera and the inspected item in the world coordinate system, and the second pose parameter of the camera, a predicted image that should be obtained by the camera when shooting the inspected item at the second pose parameter can be simulated using computer graphics or view synthesis techniques.

[0067] Optionally, a three-dimensional model of the inspected item constructed according to each target video frame, or a three-dimensional model of the inspected item pre-stored in the picture inspection system, is used as the geometric representation of the inspected item. According to the second pose parameter and the camera intrinsic parameter, a predicted image is generated by a rendering algorithm such as ray casting or neural radiance field.

[0068] Exemplarily, taking the ray casting algorithm as an example, a blank image is generated according to a preset image resolution, and then each pixel point (u, v) in the blank image is traversed. A direction vector of each pixel point in the camera coordinate system is calculated according to the camera intrinsic parameter :

[0069]

[0070] wherein, is the intrinsic matrix of the camera. Assuming that the center of the camera lens is used as the camera optical center , a set of rays can be obtained from the camera optical center and the direction vector of each pixel in the camera coordinate system . Then, according to the second pose parameter of the camera, such as the second rotation matrix and the second translation vector , the camera optical center and the direction vector of each pixel point in the camera coordinate system are converted to the world coordinate system to obtain a set of world coordinate system rays . Wherein, , .

[0071] Next, according to the second shooting distance d2, the closest intersection point pi of each world coordinate system ray and the three-dimensional model of the inspected item is calculated to obtain a set of intersection points.

[0072] It can be understood that the material parameters of the inspected object can be obtained from the three-dimensional model, and since the light condition inside the inspection room changes little, the light parameters such as ambient light, diffuse reflection and specular reflection can be simulated according to the light environment of the inspection room. Then, according to the above light parameters and material parameters, the color parameters of each intersection point of the intersection set are determined through a light model such as the Phong light model. Each pixel point (u, v) corresponds to a world coordinate system ray emitted from the camera optical center, and the color parameter of the intersection point pi of the three-dimensional model of the inspected object is the color of the pixel (u, v). Finally, the predicted shooting image is rendered.

[0073] In step S40, the parameter error of the first pose parameter and the second pose parameter is determined, and the image error of the to-be-inspected picture and the shooting picture is determined. When the parameter error and the image error are both less than a preset error threshold, it is determined that the to-be-inspected picture passes the inspection.

[0074] In the embodiment, the picture inspection system reads the time label of the to-be-inspected picture and immediately goes to the target video frame corresponding to the time label in the inspection room video. Then, the two-dimensional feature points of the inspected object in the to-be-inspected picture are matched with the three-dimensional model of the inspected object by using the target video frames, and the first pose information of the camera during shooting is inversely deduced. Then, the picture inspection system calculates the second pose information of the camera in the world coordinate system of the inspection room by using the preset feature points of the camera recognized in the video, and renders the shooting image that should be shot by the camera under the second pose by using computer graphics technology. If the user temporarily replaces the inspected object with another qualified product or uses a "qualified picture" to impersonate, the rendered shooting image will be obviously different from the to-be-inspected picture. Finally, the pose parameter difference and the image pixel difference of the camera are compared. Only when both errors are lower than the set threshold, it is determined that the picture is actually shot in real time on site, and the inspection can pass.

[0075] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as the above embodiment one can be referred to the above introduction, and will not be described in detail. On this basis, the first pose parameter includes a first rotation matrix and a first translation vector, and the first rotation matrix and the first translation vector are determined by matching the two-dimensional feature points of the inspected object with the three-dimensional model of the inspected object, and the second pose parameter includes a second rotation matrix and a second translation vector, and the second rotation matrix and the second translation vector are determined by matching the preset feature points of the camera with the three-dimensional model of the camera. Figure 2 , step S20 includes steps S21-S23:

[0076] In step S21, the texture feature of the inspected object is recognized to obtain the two-dimensional feature points of the inspected object.

[0077] Exemplarily, a multi-scale representation of the to-be-inspected picture is constructed, and the to-be-inspected picture is subjected to multiple Gaussian blur down-sampling to form a multi-group multi-layer pyramid structure. Successive layers are generated in each group by different Gaussian kernel blurring, and the groups are connected by down-sampling. The adjacent two layers in the same group in the Gaussian pyramid are subtracted to obtain a difference image, and all the difference images constitute a difference pyramid. For each pixel point in the difference pyramid, the gray values of the pixel point and adjacent pixel points in the same layer and adjacent pixel points in the upper and lower two layers are compared. If the gray value of the pixel point is greater than or less than all the adjacent pixel points, the pixel point is marked as a two-dimensional feature point. Then, a gradient direction histogram of the pixel points in the neighborhood of the two-dimensional feature point is calculated, a main direction is specified for each two-dimensional feature point, and a descriptor vector is generated according to the gradient direction histogram.

[0078] In step S22, the two-dimensional feature point is matched with the three-dimensional model of the inspected object to obtain a three-dimensional feature point corresponding to the two-dimensional feature point.

[0079] In step S23, a first rotation matrix and a first translation vector of the camera are determined according to a projection relationship between the two-dimensional feature point and the three-dimensional feature point and the camera intrinsic parameter.

[0080] Exemplarily, the two-dimensional descriptor vector obtained above is compared with a texture descriptor pre-stored in the three-dimensional model, a three-dimensional feature point of the inspected object corresponding to the two-dimensional feature point is screened out by cosine similarity, and a matching point pair is formed. According to the projection relationship of the matching point pair and the camera intrinsic parameter, a first rotation matrix and a first translation vector of the camera are determined.

[0081] Optionally, step S23 further includes steps S231-S232.

[0082] In step S231, the two-dimensional feature point, the three-dimensional feature point, and the camera intrinsic parameter are input into a preset EPnP (Efficient Perspective-n-Point) model, the three-dimensional feature point is represented as a weighted sum of control points by the EPnP model, and the coordinates of the control points in the camera coordinate system are determined according to the projection relationship and the camera intrinsic parameter.

[0083] In step S232, the first rotation matrix and the first translation vector of the camera are determined according to the coordinates of the control points in the camera coordinate system and the coordinates of the control points in the world coordinate system.

[0084] Exemplarily, the control points are obtained by weighted centroid decomposition from the three-dimensional feature points, and all the three-dimensional feature points are represented by a linear combination of the control points. Then, a projection equation capable of representing the projection of the control points into a two-dimensional image is established according to the camera intrinsic parameter and the projection relationship, and the coordinates of the control points in the camera coordinate system are obtained by solving the projection equation.

[0085] For example, suppose the three-dimensional feature points of the inspected item The corresponding two-dimensional feature points are The 3D feature points are decomposed using weighted centroid decomposition and represented as a weighted sum of control points: = ,in, As weight, Let be the coordinates of the control points in the world coordinate system, and n represent the number of control points. Then, based on the camera intrinsic parameters and the above projection relationship, a projection equation is established to represent projecting the control points onto the two-dimensional image: ,in, K is the depth scaling factor; K is the camera intrinsic parameter matrix. These are the coordinates of the control points in the camera coordinate system. These are the coordinates of the 3D feature points in the camera coordinate system.

[0086] Optionally, the solution process for the above projection equation is as follows: first, expand the above projection equation to eliminate the depth scale factor. For example, taking the x and y components in the coordinate system (x, y, z) as an example, the projection equation is expanded to eliminate the depth scale factor. The following system of equations is obtained:

[0087]

[0088] Eliminating the denominators from the above system of equations yields two linear equations:

[0089]

[0090] Will Substitute and get about The linear equations of all control points are stacked to form a matrix: Mx=0. Where x= M is The corresponding coefficient matrix. Since the equation is homogeneous and contains noise, M=UΣ can be solved using the least squares method. Take x as the right singular vector corresponding to the smallest singular value of V (right singular vector), that is, the last column of V, to obtain the coordinates of the control point in the camera coordinate system.

[0091] Obtain the coordinates of the control points in the camera coordinate system Then, based on its coordinates in the world coordinate system Solve for the first rotation matrix R1 and the first translation vector t1: .

[0092] It can be understood that, in the cosmetic inspection scene, the placement position of the inspected item on the inspection table is generally fixed, so the coordinates of the three-dimensional feature points of the inspected item in the world coordinate system, i.e., the coordinates of the control points in the world coordinate system, are known. In some special embodiments, the coordinates of the inspected item can also be calibrated in advance to determine the coordinates of the three-dimensional feature points of the inspected item in the world coordinate system.

[0093] In another possible implementation, the first pose parameter further includes a first shooting distance, and the step S20 further includes steps S24-S26:

[0094] In step S24, a pixel size of the inspected item is determined according to the two-dimensional feature points of the inspected item.

[0095] In step S25, an actual size of the inspected item is determined according to the three-dimensional feature points of the inspected item.

[0096] In step S26, the first shooting distance of the camera is determined according to the pixel size of the inspected item, the actual size of the inspected item, and the camera intrinsic parameter.

[0097] For example, the Euclidean distance between the two-dimensional feature points of the inspected item in the to-be-inspected picture is determined to obtain the pixel size of the inspected item. The Euclidean distance between the three-dimensional feature points in the three-dimensional model of the inspected item is determined to obtain the actual size of the inspected item.

[0098] According to the principle of similar triangles, let the Euclidean distance between the two two-dimensional feature points in the to-be-inspected picture be d pixel, the Euclidean distance between the corresponding three-dimensional feature points be D real, and the focal length of the camera be f. Then the first shooting distance, i.e., the distance from the inspected item in the picture to the camera, is Z=D real f / d pixel.

[0099] Based on the above embodiments of the present application, in the third embodiment of the present application, the same or similar contents as the above embodiments can be referred to the above introduction, and will not be described hereinafter. On this basis, the second pose parameter includes a second rotation matrix and a second translation vector, and the step S30 includes steps S31-S34: Figure 3

[0100] In step S31, pixel coordinates of a preset feature point of the camera in each target video frame are determined.

[0101] In step S32, the preset feature point is projected to a world coordinate system according to the pixel coordinates and the camera parameter corresponding to each target video frame, to obtain a first projection equation of each preset feature point.

[0102] ​Exemplarily, assuming that the pixel coordinates of the preset feature point of the camera in the target video frame corresponding to the time tag are In the target video frame of camera i, the pixel coordinates are According to the collinearity principle of camera imaging, the pixel coordinates can be projected to the world coordinate system to obtain the coordinates of the preset feature point in the world coordinate system Thus, the second rotation matrix and the second translation vector of the camera can be determined according to the PnP (Perspective-n-Point) algorithm according to the coordinates of the preset feature point in the world coordinate system and the coordinates in the camera coordinate system.

[0103] Optionally, step S32 comprises steps S321-S322:

[0104] In step S321, the preset feature point is projected to the camera coordinate system according to the pixel coordinates and the camera intrinsic parameters corresponding to each target video frame to obtain the second projection equation of each preset feature point.

[0105] Exemplarily, assuming that the pixel coordinates of the preset feature point of the camera in the i-th target video frame are Then, according to the camera intrinsic parameters corresponding to the i-th target video frame , the preset feature point is projected to the camera coordinate system, the two-dimensional pixel coordinates are converted to the three-dimensional ray direction in the camera coordinate system, and the second projection equation is obtained:

[0106]

[0107] wherein, represents the depth scale factor of the preset feature point j in the camera i coordinate system.

[0108] In step S322, the preset feature point is projected to the world coordinate system according to the second projection equation of each preset feature point and the camera extrinsic parameters corresponding to each target video frame to obtain the first projection equation of each preset feature point.

[0109] Exemplarily, the three-dimensional preset feature point in the camera coordinate system can be expressed as .Then, according to the camera extrinsic parameters, i.e., the rotation matrix and the translation vector , the three-dimensional preset feature point in the camera coordinate system is converted to the three-dimensional feature point in the world coordinate system to obtain the first projection equation: . ​​​

[0110] Step S33, determining the solution of the second projection equation according to the least square method, to obtain the coordinates of each preset feature point in the world coordinate system.

[0111] It can be understood that, for the same preset feature point j, the observation of different cameras i will provide different ray directions . Then, the above first projection equation corresponding to at least two cameras can be solved simultaneously to eliminate and solve .

[0112] Step S34, determining the second rotation matrix and the second translation vector of the camera according to the coordinates of each preset feature point in the world coordinate system and the coordinates in the camera coordinate system.

[0113] After obtaining the coordinates of the preset feature points in the world coordinate system and the coordinates in the camera coordinate system, the second rotation matrix and the second translation vector of the camera can be determined according to the PnP algorithm.

[0114] Optionally, the second pose parameter can further include a second shooting distance, and after the step S34, the picture authenticity verification method further includes steps S35-S36:

[0115] Step S35, determining the coordinates of the camera in the world coordinate system according to the coordinates of the preset feature points in the world coordinate system.

[0116] Step S36, determining the second shooting distance according to the coordinates of the camera in the world coordinate system and the coordinates of the inspected object in the world coordinate system.

[0117] For example, according to the coordinates of the preset feature points on the camera in the world coordinate system, the coordinates of the geometric center of each preset feature point in the world coordinate system are determined and taken as the coordinates of the camera in the world coordinate system. Similarly, the coordinates of the geometric center of the three-dimensional model of the inspected object in the world coordinate system are taken as the coordinates of the inspected object in the world coordinate system. Then, the Euclidean distance between the coordinates of the camera in the world coordinate system and the coordinates of the inspected object in the world coordinate system is taken as the second shooting distance.

[0118] Based on the above embodiments of the present application, in the fourth embodiment of the present application, the same or similar contents as the above embodiments can be referred to the above introduction, and the subsequent will not be repeated. On this basis, referring to Figure 4 , after step S10, the picture authenticity verification method further includes steps S50-S60:

[0119] Step S50, determining the similarity between the inspected object in the target video frame and the inspected object in the picture to be verified.

[0120] Step S60, when the similarity is greater than a preset similarity threshold, performing the step of determining the feature points of the target video frame and the target video frame.

[0121] For example, the similarity of the texture features and the contour features of the target video frame and the target video frame is compared. If the similarity is less than a preset similarity threshold, it indicates that the target video frame and the target video frame are not the same object, and the authenticity verification of the target video frame fails. When the similarity is greater than the preset similarity threshold, the step of determining the feature points of the target video frame and the target video frame is performed to further determine the first pose parameter of the corresponding camera of the target video frame and the second pose parameter of the corresponding camera of the target video frame, and then verify whether the target video frame is a real picture taken in the test room according to the similarity of the first pose parameter and the second pose parameter.

[0122] It should be noted that the above examples are only used to understand the present application and do not limit the picture authenticity verification method of the present application. More forms of simple transformation based on this technical concept are within the protection scope of the present application.

[0123] The present application provides a picture authenticity verification device, which comprises at least one processor and a memory connected with the at least one processor. The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the picture authenticity verification method in the above embodiment one.

[0124] Reference will be made to the following description of the embodiments of the present application with reference to the accompanying drawings. Figure 5 which shows a structural schematic diagram of a picture authenticity verification device suitable for implementing the embodiments of the present application. The picture authenticity verification device in the embodiments of the present application can include but is not limited to mobile terminals such as notebook computers, tablet computers (PAD, Portable Application Description) and the like, and fixed terminals such as digital TVs, desktop computers and the like. Figure 5 The picture authenticity verification device shown is only an example and should not limit the functions and use range of the embodiments of the present application.

[0125] As Figure 5As shown, the picture authenticity verification device can include a processing device 1001 (e.g., a central processing unit, a graphics processing unit, etc.) that can perform various appropriate actions and processes according to programs stored in a read-only memory (ROM) 1002 or programs loaded from a storage device 1003 into a random access memory (RAM) 1004. Various programs and data required for the picture authenticity verification device to operate are also stored in the random access memory 1004. The processing device 1001, the read-only memory 1002, and the random access memory 1004 are connected to each other by a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: input devices 1007 including, for example, a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; the storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the picture authenticity verification device to communicate with other devices wirelessly or by wire to exchange data. Although the picture authenticity verification device with various systems is shown in the figure, it should be understood that all the shown systems are not required to be implemented or possessed. More or fewer systems can be alternatively implemented or possessed.

[0126] In particular, according to embodiments of the present disclosure, the processes described above with reference to the flowcharts can be implemented as a computer software program. For example, embodiments of the present disclosure include a computer program product comprising a computer program carried on a computer readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network by a communication device, or installed from the storage device 1003, or installed from the read-only memory 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the methods of the embodiments of the present disclosure are performed.

[0127] The picture authenticity verification device provided by the present disclosure adopts the picture authenticity verification method in the above-mentioned embodiments, and can solve the technical problem that the current image verification method cannot verify the authenticity of data. Compared with the prior art, the picture authenticity verification device provided by the present disclosure has the same beneficial effects as the picture authenticity verification method provided by the above-mentioned embodiments, and other technical features in the picture authenticity verification device are the same as the features disclosed in the previous embodiment method, which will not be repeated here.

[0128] It should be understood that various parts of the present application can be realized with hardware, software, firmware, or a combination thereof. In the above description of embodiments, specific functional configurations, structures, materials or characteristics can be combined in any appropriate manner in one or more embodiments or examples.

[0129] The above description is merely illustrative of the application and the scope of the application should not be limited thereto as modifications within the scope of the application will occur to those skilled in the art upon reading the description of the application. Accordingly, all modifications and equivalents thereof are intended to be included within the scope of the application. Therefore, the scope of the application should be determined by the following claims.

[0130] The present application provides a computer readable storage medium having stored thereon computer readable program instructions (i.e. computer programs) for performing the picture authenticity verification method in the above embodiments.

[0131] The computer readable storage medium provided by the present application may, for example, be a U disk, but is not limited to an electric, magnetic, optical, electromagnetic, infrared or semiconductor system, system or device, or any combination of the above. More specific examples of the computer readable storage medium can include, but are not limited to, an electric connection with one or more conductive wires, a portable computer disk, a hard disk, a random access memory (RAM), a read only memory (ROM), an erasable programmable read only memory (EPROM) or a flash memory, an optical fiber, a portable compact disk read only memory (CD-ROM), an optical storage device, a magnetic storage device, or any appropriate combination of the above. In the present embodiment, the computer readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system or device. The program code contained on the computer readable storage medium can be transmitted by any appropriate medium, including but not limited to an electric wire, an optical cable, a radio frequency (RF), etc., or any appropriate combination of the above.

[0132] The above computer readable storage medium can be contained in the picture authenticity verification device; or can exist separately without being assembled into the picture authenticity verification device.

[0133] The computer-readable storage medium described above can bear one or more programs, when the one or more programs are executed by the picture authenticity verification device, the picture authenticity verification device can be programmed in one or more programming languages or combinations thereof to execute computer program codes for performing the operations of the present application, the programming languages include object-oriented programming languages such as Java, Smalltalk, C++, and conventional procedural programming languages such as "C" language or similar programming languages. The program code can be executed entirely on the user computer, partially on the user computer, or as a separate software package, partially on the user computer and partially on a remote computer, or entirely on a remote computer or server. In the case involving a remote computer, the remote computer can be connected to the user computer through any kind of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (for example, through the Internet by using an Internet service provider).

[0134] The flowcharts and block diagrams in the drawings illustrate the possible implementation architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each block in the flowcharts or block diagrams can represent a module, a program segment or a part of code containing one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions noted in the blocks can occur in different order than that noted in the drawings. For example, two blocks represented in succession can actually be executed substantially in parallel, and sometimes they can be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and the combination of blocks in the block diagrams and / or flowcharts, can be implemented by a dedicated hardware-based system that performs the specified functions or operations, or can be implemented by a combination of dedicated hardware and computer instructions.

[0135] The modules involved in the embodiments described in the present application can be implemented in software or in hardware. In some cases, the name of the module does not constitute a limitation on the unit itself.

[0136] The readable storage medium provided by the application is a computer readable storage medium, which stores computer readable program instructions (i.e. computer program) for executing the picture authenticity verification method described above, and can solve the technical problem that the current image verification method cannot verify the authenticity of data. Compared with the prior art, the computer readable storage medium provided by the application has the same beneficial effects as the picture authenticity verification method provided by the above-mentioned embodiments, and will not be described here.

[0137] The application also provides a computer program product comprising a computer program, which, when executed by a processor, implements the steps of the picture authenticity verification method as described above.

[0138] The computer program product provided by the application can solve the technical problem that the current image verification method cannot verify the authenticity of data. Compared with the prior art, the computer program product provided by the application has the same beneficial effects as the picture authenticity verification method provided by the above-mentioned embodiments, and will not be described here.

[0139] The above-mentioned is only part of the embodiments of the application, and does not limit the patent scope of the application, and any equivalent structural transformation made by using the content of the application specification and drawings, or direct / indirect application in other related technical fields is included in the patent protection scope of the application.

Claims

1. A method for verifying the authenticity of an image, characterized in that, The method for verifying the authenticity of the images includes the following steps: Based on the time tag of the image to be inspected, obtain the target video frame corresponding to the time tag from the inspection room video; The process involves determining the feature points of the inspected item in the target video frame and the inspected item in the image to be inspected, and determining the first pose parameter of the camera based on the feature points and camera intrinsic parameters. This includes: constructing a three-dimensional model of the inspected item using images of the inspected item from different angles in each target video frame; matching the two-dimensional feature points of the inspected item in the image to be inspected with the three-dimensional model of the inspected item to determine the three-dimensional feature points corresponding to the two-dimensional feature points; and calculating the first pose parameter of the camera based on the two-dimensional feature points, the corresponding three-dimensional feature points, and the camera intrinsic parameters. The process of determining preset feature points of the camera in the target video frame, determining the second pose parameters of the camera based on the preset feature points and camera parameters, and predicting the captured image based on the second pose parameters includes: converting the two-dimensional pixel coordinates of the preset feature points detected in different target video frames into three-dimensional coordinates in the world coordinate system, and then determining the second pose parameters of the camera based on the coordinates of the preset feature points in the world coordinate system and the coordinates in the camera coordinate system, wherein the preset feature points are non-coplanar points pre-marked on the camera; The parameter error between the first pose parameter and the second pose parameter, and the image error between the image to be inspected and the captured image are determined. When both the parameter error and the image error are less than a preset error threshold, the image to be inspected is determined to have passed inspection.

2. The image authenticity verification method as described in claim 1, characterized in that, The first pose parameter includes a first rotation matrix and a first translation vector. The step of determining the feature points of the inspected item in the target video frame and the inspected item in the image to be inspected, and determining the first pose parameter of the camera based on the feature points and camera intrinsic parameters, includes: The texture features of the inspected item are identified to obtain the two-dimensional feature points of the inspected item; The two-dimensional feature points are matched with the three-dimensional model of the inspected item to obtain the three-dimensional feature points corresponding to the two-dimensional feature points; Based on the projection relationship between the two-dimensional feature points and the three-dimensional feature points, and the camera intrinsic parameters, the first rotation matrix and the first translation vector of the camera are determined.

3. The image authenticity verification method as described in claim 2, characterized in that, The step of determining the first rotation matrix and the first translation vector of the camera based on the projection relationship between the two-dimensional feature points and the three-dimensional feature points, and the camera intrinsic parameters, further includes: The two-dimensional feature points, the three-dimensional feature points, and the camera intrinsic parameters are input into a preset EPnP model. The three-dimensional feature points are represented as a weighted sum of control points through the EPnP model. The coordinates of the control points in the camera coordinate system are determined according to the projection relationship and the camera intrinsic parameters. Based on the coordinates of the control point in the camera coordinate system and the coordinates of the control point in the world coordinate system, determine the first rotation matrix and the first translation vector of the camera.

4. The image authenticity verification method as described in claim 1, characterized in that, The first pose parameter includes a first shooting distance. The step of determining the feature points of the inspected item in the target video frame and the inspected item in the image to be inspected, and determining the first pose parameter of the camera based on the feature points and camera intrinsic parameters, includes: The pixel size of the inspected item is determined based on the two-dimensional feature points of the inspected item; The actual size of the inspected item is determined based on its three-dimensional feature points. The first shooting distance of the camera is determined based on the pixel size of the item being inspected, the actual size of the item being inspected, and the camera's intrinsic parameters.

5. The image authenticity verification method as described in claim 1, characterized in that, The second pose parameters include a second rotation matrix and a second translation vector. The step of determining the preset feature points of the camera in the target video frame and determining the second pose parameters of the camera based on the preset feature points and camera parameters includes: Determine the pixel coordinates of the preset feature points of the camera within each of the target video frames; Based on the pixel coordinates and the camera parameters corresponding to each target video frame, the preset feature points are projected onto the world coordinate system to obtain the first projection equation of each preset feature point. The solution to the first projection equation is determined by the least squares method, and the coordinates of each of the preset feature points in the world coordinate system are obtained. The second rotation matrix and the second translation vector of the camera are determined based on the coordinates of each preset feature point in the world coordinate system and the coordinates in the camera coordinate system.

6. The image authenticity verification method as described in claim 5, characterized in that, The camera parameters include camera intrinsic parameters and camera extrinsic parameters. The step of projecting the preset feature points onto the world coordinate system based on the pixel coordinates and the camera parameters corresponding to each target video frame to obtain the first projection equation of each preset feature point includes: Based on the pixel coordinates and the camera intrinsic parameters corresponding to each target video frame, the preset feature points are projected onto the camera coordinate system to obtain the second projection equation of each preset feature point; Based on the second projection equation of each preset feature point and the camera extrinsic parameters corresponding to each target video frame, the preset feature points are projected onto the world coordinate system to obtain the first projection equation of each preset feature point.

7. The image authenticity verification method as described in claim 5, characterized in that, The second pose parameter includes a second shooting distance. After the step of determining the camera's rotation matrix and translation vector based on the coordinates of each preset feature point in the world coordinate system and its coordinates in the camera coordinate system, the method further includes: The coordinates of the camera in the world coordinate system are determined based on the coordinates of the preset feature points in the world coordinate system. The second shooting distance is determined based on the coordinates of the camera in the world coordinate system and the coordinates of the inspected item in the world coordinate system.

8. The image authenticity verification method as described in claim 1, characterized in that, After the step of obtaining the target video frame corresponding to the time tag from the laboratory video based on the time tag of the image to be inspected, the method further includes: Determine the similarity between the inspected item in the target video frame and the inspected item in the image to be inspected; When the similarity is greater than a preset similarity threshold, the step of determining the feature points of the inspected item in the target video frame and the inspected item in the image to be inspected is performed.

9. A device for verifying the authenticity of images, characterized in that, The device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the image authenticity verification method as described in any one of claims 1 to 8.

10. A storage medium, characterized in that, The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, it implements the steps of the image authenticity verification method as described in any one of claims 1 to 8.

Images