Article information checking method and system based on augmented reality technology

Abstract

The invention discloses an article information checking method and system based on an augmented reality technology, and the method comprises the steps: scanning an article through a spectral imaging device, collecting an image of the article in a spectral segment, and obtaining a multispectral reflection matrix of the article through the image of the spectral segment; extracting spectral features from a spectral section in the multispectral reflection matrix by using physical features of the article to obtain a multispectral reflection feature vector, and superposing the multispectral reflection feature vector onto an image of the article in the spectral section in real time by using color marks according to the multispectral reflection feature vector; matching the image of the superposed article in the spectral section with the image of the article before superposition in the spectral section, and rechecking the multispectral reflection feature vector according to a matching result; according to the method, the multispectral reflection feature vector is superposed on the object image through color marking, so that augmented reality display is formed; and the matching results are calculated by using the Euclidean distance, and the matching results before and after superposition are compared to complete the checking operation of the article, so that the high-precision checking of the article is realized.

Description

Technical Field

[0001] The present invention relates to the field of spectral imaging technology, and in particular to an object information verification method and system based on augmented reality technology. Background Art

[0002] As a technology that seamlessly combines virtual information with real scenes, augmented reality (AR) technology has been widely used in many industries in recent years, especially in the fields of item information verification, anti-counterfeiting detection and quality control. Combining computer vision and image processing technology, AR can intuitively superimpose virtual information on the surface of real objects, providing operators with a more convenient interactive experience. In addition, the rapid development of multispectral imaging technology has also made it possible to collect surface and internal feature information of objects in different spectral bands. Multispectral data can reveal the characteristics of objects under different lighting conditions, thereby providing physical and chemical information beyond the visible range of the naked eye. However, although these two technologies each have powerful capabilities, there are still many technical challenges in effectively combining the two in practical applications to solve complex item information verification problems.

[0003] First, existing multispectral imaging equipment is usually limited to the acquisition of a single band or a narrow range of spectral bands, making it difficult to achieve comprehensive integration of multi-band information, resulting in low accuracy in detecting features such as object details or coatings, especially on complex materials or composite objects. Second, most AR systems rely on marking and tracking under visible light, and lack effective identification methods for object features in different spectral bands, limiting their application in high-precision object verification scenarios. Third, the data processing of existing multispectral imaging technologies in different spectral bands is mostly independent processes, lacking a unified data fusion mechanism, making it difficult to ensure the reliability of detection results. In addition, external conditions such as ambient lighting have a great impact on the acquisition and subsequent processing of multispectral images, which in turn leads to deviations in verification results in scenes with large changes in lighting conditions. Summary of the invention

[0004] The purpose of this section is to summarize some aspects of embodiments of the present invention and briefly introduce some preferred embodiments. Some simplifications or omissions may be made in this section and the specification abstract and the invention title of this application to avoid blurring the purpose of this section, the specification abstract and the invention title, and such simplifications or omissions cannot be used to limit the scope of the present invention.

[0005] In view of the above existing problems, the present invention is proposed. Therefore, the present invention provides an item information verification method based on augmented reality technology to solve the problems mentioned in the background technology.

[0006] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0007] In a first aspect, the present invention provides an object information verification method based on augmented reality technology, comprising:

[0008] Scanning the object by a spectral imaging device to collect an image of the object in a spectral band, and obtaining a multispectral reflectance matrix of the object from the image of the spectral band;

[0009] Utilizing the physical characteristics of the object, extracting spectral features from the spectral segments in the multispectral reflectance matrix to obtain a multispectral reflectance feature vector, and superimposing the multispectral reflectance feature vector on the image of the object in the spectral segment in real time using a color marker according to the multispectral reflectance feature vector;

[0010] The image of the object in the spectral band after superposition is matched with the image of the object in the spectral band before superposition, and the multi-spectral reflectance feature vector is reviewed according to the matching result to complete the verification operation of the object information.

[0011] As a preferred solution of the object information verification method based on augmented reality technology described in the present invention, wherein: scanning the object by a spectral imaging device, collecting the image of the object in the spectral range, and obtaining the spectral reflectance matrix of the object from the image of the spectral range, including:

[0012] The spectrum is divided into short-wave infrared light, near-infrared light and ultraviolet light, and a grayscale image matrix with a resolution of 1280×720 is generated according to the spectrum. The spectral reflectance matrix of the object is obtained through the reflectance value of each pixel in the grayscale image matrix.

[0013] As a preferred solution of the method for checking item information based on augmented reality technology of the present invention, it also includes:

[0014] In the process of generating the spectral reflectance matrix, the light intensity under the standard environment is set to L std , and collect the light intensity L in the current environment in real time env ;

[0015] If the light intensity L in the current environment is collected env The light intensity L under the standard environment is set std If they are inconsistent, the reflectivity value of each pixel in the grayscale image matrix is ​​corrected using the light intensity correction formula.

[0016] As a preferred solution of the object information verification method based on augmented reality technology described in the present invention, wherein: utilizing the physical characteristics of the object, extracting spectral features from the spectral segments in the multispectral reflectance matrix to obtain a multispectral reflectance feature vector includes:

[0017] For short-wave infrared light, based on the surface structure and material coating of the object, extract the average reflectivity of the surface material of the object under short-wave infrared light.

[0018] For near-infrared light, based on the chemical composition of the surface of the object, extract the maximum reflectivity of the surface material of the object under near-infrared light.

[0019] For ultraviolet light, the absorption rate of the object is extracted based on the ultraviolet absorption characteristics of the object's surface.

[0020] A multi-spectral reflectance feature vector is obtained according to the average reflectance, the maximum reflectance and the absorbance.

[0021] As a preferred solution of the object information verification method based on augmented reality technology described in the present invention, wherein: according to the multi-spectral reflection feature vector, the multi-spectral reflection feature vector is superimposed on the image of the object in the spectral segment in real time using color marking, including:

[0022] If the maximum reflectivity in the near-infrared light is within the allowable range of the material of the object, that is, the standard value deviation range of the material of the object is ±0.3, it will be marked as "green", indicating that the current object meets expectations, otherwise no mark will be made;

[0023] If the average reflectivity of short-wave infrared light deviates from the standard range [0.5, 0.8], it will be marked as "yellow", indicating that there is a coating missing or uneven structure on the surface of the object, otherwise no mark will be made;

[0024] If the UV light absorption rate deviates from the standard range [0.75, 0.85], it will be marked as "red", indicating that there is material aging or abnormal anti-counterfeiting mark on the surface of the item, otherwise no mark will be made.

[0025] As a preferred solution of the object information verification method based on augmented reality technology described in the present invention, the image of the object in the spectral band after superposition is matched with the image of the object in the spectral band before superposition, including:

[0026] Define the multi-spectral reflectance feature vector of the superimposed object in the spectral band image as F = The multispectral reflectance feature vector of the object in the spectral band image before superposition is defined as

[0027] Perform matching calculation on F and F0 by using Euclidean distance to obtain a matching result S(F, F0);

[0028] Among them, Flag represents the number of color marks in the multispectral reflectance feature vector of the object in the image of the spectral segment after superposition, and Flag0 represents the number of color marks in the multispectral reflectance feature vector of the object in the image of the spectral segment before superposition.

[0029] As a preferred solution of the object information verification method based on augmented reality technology described in the present invention, wherein: reviewing the multi-spectral reflectance feature vector according to the matching result to complete the verification operation of the object information includes:

[0030] When the matching result S(F,F0)<0.05, it indicates that the item has passed the verification;

[0031] When the matching result 0.05≤S(F,F0)<0.1, it indicates that the object has a slight deviation, and the spectral features in the multi-spectral reflectance feature vector can be fine-tuned and the matching result S(F,F0) can be recalculated;

[0032] When S(F, F0)≥0.1, it is displayed that the item has failed the verification, the spectral feature with the most serious deviation is highlighted, the most serious spectral feature is removed, the multi-spectral reflectance feature vector is updated, and the matching result S(F, F0) is recalculated.

[0033] In a second aspect, the present invention provides an item information verification system based on augmented reality technology, which includes:

[0034] The multispectral image acquisition module is configured to scan the object through a spectral imaging device, collect images of the object in a spectral range, and obtain a multispectral reflectance matrix of the object from the images of the spectral range;

[0035] The spectral feature extraction and augmented reality display module is configured to extract spectral features from the spectral segments in the multi-spectral reflectance matrix using physical features of the object to obtain a multi-spectral reflectance feature vector, and to superimpose the multi-spectral reflectance feature vector on the image of the object in the spectral segment in real time using a color marker according to the multi-spectral reflectance feature vector;

[0036] The multispectral image matching and verification module is configured to match the image of the object in the spectral segment after superposition with the image of the object in the spectral segment before superposition, review the multispectral reflection feature vector according to the matching result, and complete the verification operation of the object information.

[0037] In a third aspect, the present invention provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, wherein: the processor implements any step of the above method when executing the computer program.

[0038] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein: the computer program implements any step of the above method when executed by a processor.

[0039] Compared with the prior art, the invention has the following beneficial effects:

[0040] 1. The present invention scans the object through the spectrum bands of short-wave infrared, near-infrared and ultraviolet light, so that the reflection characteristics of the object in different spectrum bands can be fully displayed, so that the system can accurately detect the tiny features and coating status of the surface of the object, improve the detection accuracy, and solve the problem of insufficient multi-band integration capability of the prior art;

[0041] 2. By combining the feature vector of multispectral imaging with augmented reality technology, the features in different spectral bands are superimposed on the object image in real time using color markers, breaking the limitation that most AR technologies rely on visible light, so that the object information in multispectral bands can be presented intuitively, greatly improving the accuracy and applicability of object verification;

[0042] 3. By calculating the Euclidean distance, the multispectral reflectance feature vector and the standard feature vector can be effectively matched to ensure the consistency of the item verification results. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. Among them:

[0044] Figure 1 This is an overall flow chart of an object information verification method based on augmented reality technology according to an embodiment of the present invention;

[0045] Figure 2 This is a matching deviation and loss curve diagram of an object information verification method based on augmented reality technology described in an embodiment of the present invention. DETAILED DESCRIPTION

[0046] In order to make the above-mentioned purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the drawings of the specification. Obviously, the described embodiments are part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary persons in the art without creative work should fall within the scope of protection of the present invention.

[0047] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0048] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The term "in one embodiment" that appears in different places in this specification does not necessarily refer to the same embodiment, nor does it refer to a separate or selective embodiment that is mutually exclusive with other embodiments.

[0049] The present invention is described in detail with reference to schematic diagrams. When describing the embodiments of the present invention, for the sake of convenience, the cross-sectional diagrams showing the device structure will not be partially enlarged according to the general scale, and the schematic diagrams are only examples, which should not limit the scope of protection of the present invention. In addition, in actual production, the three-dimensional dimensions of length, width and depth should be included.

[0050] At the same time, in the description of the present invention, it should be noted that the directions or positional relationships indicated by the terms "upper, lower, inner and outer" are based on the directions or positional relationships shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific direction, be constructed and operated in a specific direction, and therefore cannot be understood as limiting the present invention. In addition, the terms "first, second or third" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance.

[0051] In the present invention, unless otherwise clearly specified and limited, the terms "install, connect, connect" should be understood in a broad sense, for example: it can be a fixed connection, a detachable connection or an integral connection; it can also be a mechanical connection, an electrical connection or a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0052] Example 1

[0053] Reference Figure 1 , which is the first embodiment of the present invention, and provides an object information verification method based on augmented reality technology, comprising:

[0054] S1. Scanning the object by a spectral imaging device to collect the image of the object in the spectral band, and obtaining the multi-spectral reflectance matrix of the object from the image of the spectral band;

[0055] Furthermore, the spectrum is divided into short-wave infrared light (900nm-950nm), near infrared light (750nm-780nm) and ultraviolet light (300nm-320nm). A grayscale image matrix with a resolution of 1280×720 is generated according to the spectrum. The spectral reflectance matrix of the object is obtained through the reflectance value of each pixel in the grayscale image matrix.

[0056] Specifically, the spectral reflectance matrix M of the object is defined as:

[0057]

[0058] Among them, R mn represents the reflectance of the mth spectral segment at the nth pixel, λ represents the spectral wavelength, and x,y represents the position of the pixel;

[0059] It should be noted that in the process of generating the spectral reflectance matrix, the ambient light intensity will affect the accuracy of the reflectance measurement, so the final generated spectral reflectance matrix needs to be corrected;

[0060] Furthermore, in the process of generating the spectral reflectance matrix, the light intensity under the standard environment is set to L std , and collect the light intensity L in the current environment in real time env ;

[0061] Furthermore, if the light intensity L in the current environment is collected env The light intensity L under the standard environment is set std If they are inconsistent, the reflectivity value of each pixel in the grayscale image matrix is ​​corrected by the light intensity correction formula;

[0062] Specifically, the defined light intensity correction formula is expressed as:

[0063]

[0064] Wherein, M′ is the spectral reflectance matrix M of the corrected object;

[0065] For example, assuming that L env =300lx, L std =500lx, the correction factor is Then all spectral wavelengths in the spectral reflectance matrix are amplified according to this ratio;

[0066] It should be explained that lx is expressed as a unit (lux), which is a physical term used to indicate the intensity of light and the amount of illumination on the surface area of ​​an object;

[0067] Furthermore, the defined light intensity correction formula is expanded to obtain:

[0068]

[0069] Among them, M′ (i) (x, y, λ) is the spectral reflectance matrix M of the object after i corrections; N is the total number of corrections;

[0070] Furthermore, each corrected spectral reflectance matrix is ​​stored in a two-dimensional table. By calling the two-dimensional table, the spectral reflectance matrix after each correction can be observed, and by summing all the records in the table, the total number of times the spectral reflectance matrix in the two-dimensional table is corrected can be obtained.

[0071] It should be noted that, through the total number of times the spectral reflectance matrix in the two-dimensional table is corrected, the light intensity under the standard environment can be fine-tuned to control the influence of the light intensity under the current environment on the object, thereby reducing the number of corrections of the spectral reflectance matrix M and improving the efficiency of object detection;

[0072] S2, using the physical characteristics of the object, extracting spectral features from the spectral segments in the multispectral reflectance matrix to obtain a multispectral reflectance feature vector, and superimposing the multispectral reflectance feature vector on the image of the object in the spectral segment in real time using a color marker according to the multispectral reflectance feature vector;

[0073] Furthermore, for short-wave infrared light, the average reflectivity of the surface material of the object under short-wave infrared light is extracted according to the surface structure and material coating of the object (such as uniformly coated ceramic coating, material with uneven surface coating or inconsistent thickness).

[0074] Furthermore, for near-infrared light, according to the chemical composition of the surface of the object material (such as polymer material polyethylene or metal material aluminum or copper), the maximum reflectivity of the surface material of the object under near-infrared light is extracted.

[0075] Furthermore, for ultraviolet light, according to the ultraviolet absorption characteristics of the surface of the object (such as fluorescent coating or special ink and the intensity of the irradiated light, such characteristics will emit a fluorescent signal of a specific wavelength after absorbing ultraviolet light and will show a higher absorption rate), the absorption rate of the extracted object

[0076] Furthermore, according to the average reflectivity Maximum reflectivity And the absorption rate Obtaining a multispectral reflectance feature vector;

[0077] Furthermore, if the maximum reflectivity in the near-infrared light is within the allowable range of the material of the object, that is, the standard value deviation range of the material of the object is ±0.3, it is marked as "green", indicating that the current object meets expectations, otherwise no mark is made;

[0078] Furthermore, if the average reflectivity of short-wave infrared light deviates from the standard range [0.5, 0.8], it is marked as “yellow”, indicating that there is a coating missing or uneven structure on the surface of the object, otherwise no mark is made;

[0079] Furthermore, if the UV light absorption rate deviates from the standard range [0.75, 0.85], it will be marked as "red", indicating that the material on the surface of the item is aged or the anti-counterfeiting mark is abnormal, otherwise no mark will be made;

[0080] Specifically, the unmarked multispectral reflection feature vector is represented as the image of the object in the spectral segment before superposition, and the marked multispectral reflection feature vector is represented as the image of the object in the spectral segment after superposition;

[0081] It should be noted that, since the color tags are independent tags for different spectral features, the results of each spectral feature are separated in the augmented reality display, and the three colors will not be displayed simultaneously in the same object area;

[0082] S3, matching the image of the object in the spectral segment after superposition with the image of the object in the spectral segment before superposition, reviewing the multi-spectral reflectance feature vector according to the matching result, and completing the verification operation of the object information;

[0083] Furthermore, the multispectral reflectance feature vector of the superimposed object in the spectral band image is defined as The multispectral reflectance feature vector of the object in the spectral band image before superposition is defined as

[0084] Furthermore, the Euclidean distance is used to match F and F0, and the matching result S(F,F0) is obtained;

[0085] Specifically, the Euclidean distance calculation formula is expressed as:

[0086]

[0087] Among them, Flag represents the number of color marks in the multispectral reflectance feature vector of the object in the image of the spectral segment after superposition, and Flag0 represents the number of color marks in the multispectral reflectance feature vector of the object in the image of the spectral segment before superposition;

[0088] It should be noted that the difference between the features of the items is quantified by calculating the Euclidean distance to determine whether the current information of the item after processing is consistent with the information of the item before processing;

[0089] Furthermore, the consistency of item information is defined by the following rules:

[0090] Rule 1: When the matching result S(F,F0)<0.05, the item is shown to have passed the verification;

[0091] Rule 2: When the matching result 0.05≤S(F,F0)<0.1, it indicates that the object has a slight deviation, and the spectral features in the multi-spectral reflectance feature vector can be fine-tuned and the matching result S(F,F0) can be recalculated;

[0092] It should be noted that allowing fine-tuning of the spectral features in the multispectral reflectance feature vector can give the system flexibility in handling slight object deviations, avoid failures in object information verification due to minor deviations, and improve the system's fault tolerance;

[0093] Rule 3: When S(F, F0) ≥ 0.1, the item is shown as unchecked, the spectral feature with the most serious deviation is highlighted, the most serious spectral feature is removed, the multi-spectral reflectance feature vector is updated, and the matching result S(F, F0) is recalculated;

[0094] It should be noted that by eliminating the most serious spectral features, the error propagation in the system is reduced, thereby improving the accuracy of item information verification.

[0095] Furthermore, this embodiment also provides an item information verification system based on augmented reality technology, including:

[0096] The multispectral image acquisition module is configured to scan the object through a spectral imaging device, collect images of the object in a spectral range, and obtain a multispectral reflectance matrix of the object from the images of the spectral range;

[0097] The spectral feature extraction and augmented reality display module is configured to extract spectral features from the spectral segments in the multi-spectral reflectance matrix using physical features of the object to obtain a multi-spectral reflectance feature vector, and to superimpose the multi-spectral reflectance feature vector on the image of the object in the spectral segment in real time using a color marker according to the multi-spectral reflectance feature vector;

[0098] The multispectral image matching and verification module is configured to match the image of the object in the spectral segment after superposition with the image of the object in the spectral segment before superposition, review the multispectral reflection feature vector according to the matching result, and complete the verification operation of the object information.

[0099] This embodiment also provides a computer device, which is applicable to the case of the item information verification method based on augmented reality technology, including:

[0100] Memory and processor; the memory is used to store computer executable instructions, and the processor is used to execute computer executable instructions to implement the object information verification method based on augmented reality technology as proposed in the above embodiment.

[0101] The computer device may be a terminal, and the computer device includes a processor, a memory, a communication interface, a display screen and an input device connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, an operator network, NFC (near field communication) or other technologies. The display screen of the computer device may be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device may be a touch layer covering the display screen, or a key, trackball or touchpad provided on the housing of the computer device, or an external keyboard, touchpad or mouse, etc.

[0102] This embodiment also provides a storage medium on which a computer program is stored. When the program is executed by a processor, the object information verification method based on augmented reality technology proposed in the above embodiment is implemented.

[0103] The storage medium proposed in this embodiment and the data storage method proposed in the above embodiment belong to the same inventive concept. The technical details not fully described in this embodiment can be found in the above embodiment, and this embodiment has the same beneficial effects as the above embodiment.

[0104] Example 2

[0105] Reference Figure 2 , which is the second embodiment of the present invention, and provides an object information verification method based on augmented reality technology, comprising: verifying the accuracy of the deviation from the standard range defined in the solution of the present invention by analyzing the multi-spectral reflection characteristics of different objects, and verifying the rationality of the interval defined by the matching result;

[0106] The experiment selected multiple groups of objects of different materials (including objects A, B, and C), each group of objects contained 10 samples, and multi-spectral reflectance characteristics were tested in the short-wave infrared, near-infrared, and ultraviolet spectrum bands; the reflectance and absorptivity of each sample were averaged after multiple measurements to better reflect possible fluctuations in reality; and in order to verify the rationality of the matching interval, the experiment tested different degrees of deviation (±3%, ±5%, ±10%), and calculated the matching results of different spectral features to ensure that the defined interval is scientific. The results are shown in Table 1;

[0107] Table 1

[0108]

[0109]

[0110] From Table 1, we can see that in group A, the average reflectivity of short-wave infrared light is 0.82, which is close to the standard range (0.80, 0.90), and the deviation obtained in the matching calculation is 0.04, which is within the reasonable range of [0, 0.05]. In addition, the near-infrared reflectivity and ultraviolet absorption rate are also within the standard range, indicating that the surface structure and material properties of this group of objects are relatively uniform; in group B, the average values ​​of short-wave infrared, near-infrared and ultraviolet absorption rates are all within the standard range, especially the matching deviation is only 0.02; in group C, the short-wave infrared reflectivity of this group of objects is 0.76, which is lower than the standard range, and the ultraviolet absorption rate is also lower than the standard value of 0.68, and the matching deviation is 0.11, which exceeds the tolerance range of 0.10, indicating that the materials of group C have significant abnormalities;

[0111] By analyzing the experimental data in Table 1, it can be concluded that the matching interval defined in the present invention is reasonable, because the items A and B are within the deviation interval (0.04 and 0.02, respectively), indicating that they are basically consistent with the standard spectral features, with small deviations and in line with the standards; while the deviation of the item C is 0.11, which exceeds the matching failure threshold of 0.10, indicating that the items have serious deviations;

[0112] exist Figure 2In the figure, the matching result S(F, F0) is verified. It can be seen that when the matching deviation is less than 0.05, the loss curve shows a relatively gentle upward trend, indicating that within this deviation range, the items are basically in line with the standards within the matching result range, and the system can still effectively verify the item information; however, when the matching deviation exceeds 0.05, the loss curve rises sharply, indicating that the deviation of the spectral characteristics has exceeded the tolerable range of the system at this time, and the item verification result begins to deteriorate rapidly. The experimental results prove that it is reasonable to define the matching result range in the scheme of the present invention. Through the display of the loss curve, the rationality of the present invention has been fully verified, which can further illustrate that the method of the present invention can significantly improve the accuracy of item information verification in practical applications.

[0113] Those skilled in the art will appreciate that the embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the application can adopt the form of complete hardware embodiments, complete software embodiments, or embodiments in combination with software and hardware. Moreover, the application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program codes. The scheme in the embodiments of the present application can be implemented in various computer languages, for example, object-oriented programming language Java and literal scripting language JavaScript, etc.

[0114] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0115] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0116] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0117] Although the preferred embodiments of the present application have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.

[0118] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A method for checking item information based on augmented reality technology, characterized in that: include: Scanning the object by a spectral imaging device to collect an image of the object in a spectral band, and obtaining a multispectral reflectance matrix of the object from the image of the spectral band; Utilizing the physical characteristics of the object, extracting spectral features from the spectral segments in the multispectral reflectance matrix to obtain a multispectral reflectance feature vector, and superimposing the multispectral reflectance feature vector on the image of the object in the spectral segment in real time using a color marker according to the multispectral reflectance feature vector; The image of the object in the spectral band after superposition is matched with the image of the object in the spectral band before superposition, and the multi-spectral reflectance feature vector is reviewed according to the matching result to complete the verification operation of the object information.

2. The object information verification method based on augmented reality technology according to claim 1, characterized in that: Scanning the object by a spectral imaging device to collect an image of the object in a spectral band, and obtaining a spectral reflectance matrix of the object from the image of the spectral band, including: The spectrum is divided into short-wave infrared light, near-infrared light and ultraviolet light, and a grayscale image matrix with a resolution of 1280×720 is generated according to the spectrum. The spectral reflectance matrix of the object is obtained through the reflectance value of each pixel in the grayscale image matrix.

3. The object information verification method based on augmented reality technology as claimed in claim 2, characterized in that: Also includes: In the process of generating the spectral reflectance matrix, the light intensity under the standard environment is set to L std , and collect the light intensity L in the current environment in real time env ; If the light intensity L in the current environment is collected env The light intensity L under the standard environment is set std If they are inconsistent, the reflectivity value of each pixel in the grayscale image matrix is ​​corrected using the light intensity correction formula.

4. The object information verification method based on augmented reality technology according to claim 2 or 3, characterized in that: Utilizing the physical characteristics of the object, spectral features are extracted from the spectral segments in the multispectral reflectance matrix to obtain a multispectral reflectance feature vector, including: For short-wave infrared light, based on the surface structure and material coating of the object, extract the average reflectivity of the surface material of the object under short-wave infrared light. For near-infrared light, based on the chemical composition of the surface of the object, extract the maximum reflectivity of the surface material of the object under near-infrared light. For ultraviolet light, the absorption rate of the object is extracted based on the ultraviolet absorption characteristics of the object's surface. A multi-spectral reflectance feature vector is obtained according to the average reflectance, the maximum reflectance and the absorbance.

5. The method for verifying item information based on augmented reality technology according to claim 4, characterized in that: According to the multi-spectral reflectance feature vector, the multi-spectral reflectance feature vector is superimposed on the image of the object in the spectral segment in real time using a color marker, including: If the maximum reflectivity in near-infrared light is within the allowable range of the material of the object, that is, the standard value deviation range of the material of the object is ±0.3, it will be marked as "green", indicating that the current object meets expectations, otherwise no mark will be made; If the average reflectivity of short-wave infrared light deviates from the standard range [0.5, 0.8], it will be marked as "yellow", indicating that there is a coating missing or uneven structure on the surface of the object, otherwise no mark will be made; If the UV light absorption rate deviates from the standard range [0.75, 0.85], it will be marked as "red", indicating that there is material aging or abnormal anti-counterfeiting mark on the surface of the item, otherwise no mark will be made.

6. The method for verifying item information based on augmented reality technology according to claim 5, characterized in that: Matching the image of the object in the spectral band after superposition with the image of the object in the spectral band before superposition, including: Define the multi-spectral reflectance feature vector of the superimposed object in the spectral band image as F = The multispectral reflectance feature vector of the object in the spectral band image before superposition is defined as Perform matching calculation on F and F0 by using Euclidean distance to obtain a matching result S(F, F0); Among them, Flag represents the number of color marks in the multispectral reflectance feature vector of the object in the image of the spectral segment after superposition, and Flag0 represents the number of color marks in the multispectral reflectance feature vector of the object in the image of the spectral segment before superposition.

7. The method for verifying item information based on augmented reality technology according to claim 6, characterized in that: The multi-spectral reflectance feature vector is reviewed based on the matching result to complete the verification operation of the item information, including: When the matching result S(F,F0)<0.05, it indicates that the item has passed the verification; When the matching result 0.05≤S(F,F0)<0.1, it indicates that the object has a slight deviation, and the spectral features in the multi-spectral reflectance feature vector can be fine-tuned and the matching result S(F,F0) can be recalculated; When S(F, F0)≥0.1, it is displayed that the item has failed the verification, the spectral feature with the most serious deviation is highlighted, the most serious spectral feature is removed, the multi-spectral reflectance feature vector is updated, and the matching result S(F, F0) is recalculated.

8. An item information verification system based on augmented reality technology, based on the item information verification method based on augmented reality technology according to any one of claims 1 to 7, characterized in that: include: The multispectral image acquisition module is configured to scan the object through a spectral imaging device, collect images of the object in a spectral range, and obtain a multispectral reflectance matrix of the object from the images of the spectral range; The spectral feature extraction and augmented reality display module is configured to extract spectral features from the spectral segments in the multi-spectral reflectance matrix using physical features of the object to obtain a multi-spectral reflectance feature vector, and to superimpose the multi-spectral reflectance feature vector on the image of the object in the spectral segment in real time using a color marker according to the multi-spectral reflectance feature vector; The multispectral image matching and verification module is configured to match the image of the object in the spectral segment after superposition with the image of the object in the spectral segment before superposition, review the multispectral reflection feature vector according to the matching result, and complete the verification operation of the object information.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images