Code generation method, code generation device, program, and data collation method

Abstract

A code generation method, a code generation device, a program, and a data matching method for encoding target data such as images and audio are provided. [Solution] A code generation device 1 uses an information processing device to generate a code corresponding to the contents of target data, and the information processing unit 10 includes a sampling processing unit 20 that divides the target data into multiple sampling ranges, a feature value calculation unit 21 that calculates the average value of at least one of one or more types of element data included in each of the multiple sampling ranges and each of which is expressed numerically, for each sampling range, and an encoding processing unit 23 that generates a reference code corresponding to the target data by concatenating, as character string data, the average value for each sampling range or the numerical value of a predetermined number of digits from the average value.

Description

[Technical field]

[0001] The present invention relates to a code generation method, a code generation device, a program, and a data matching method. [Background technology]

[0002] There is a known technology that identifies the contents of target data by comparing and matching with reference data prepared in advance. A representative example of this technology is image recognition technology (see, for example, JP 2019-28985 A). However, the conventional technology requires a huge amount of learning time to obtain reference data, and the processing load required for comparison and matching with the target data is large, making high-speed processing difficult, leaving room for improvement. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2019-28985 A Summary of the Invention [Problem to be solved by the invention]

[0004] An object of a specific embodiment of the present invention is to provide a novel technique for encoding target data such as images and audio with a small amount of data. [Means for solving the problem]

[0005] [1] A data matching method according to one aspect of the present invention is a method for generating and matching a code corresponding to the contents of target data using an information processing device, the method including: (a) a first step of acquiring a reference code determined according to reference data; (b) a second step of generating a specific target code corresponding to specific target data; and (c) a third step of comparing the reference code with the specific target code to determine whether they match or do not match, and (d) the reference code in the first step is obtained by: (d1) dividing the reference data into a plurality of first sampling ranges; (d2) calculating a value of the first element data for each of the first sampling ranges for at least one type of first element data among one or more types of first element data each of which is expressed by a numerical value and which is included in each of the plurality of first sampling ranges; No. 1 determining an average value; (d3) calculating a first relative difference for each of the first sampling ranges by dividing the first average value for each of the first sampling ranges by a sum of the first average values ​​for each of the first sampling ranges; (d 4 ) each For each of the first sampling ranges First Relative Difference and (e) the identification code in the second step is determined in advance through a step of generating the reference code corresponding to the reference data by concatenating, as character string data, numerical values ​​of a predetermined number of digits from the most significant digits of the first sampling range or the first sampling range and the like, and (e) the identification code in the second step is determined in advance through a step of (e1) dividing the identification data into a plurality of second sampling ranges set in the same manner as each of the first sampling ranges, and (e2) extracting second element data included in each of the plurality of second sampling ranges, the second element data being one or more types of second element data each expressed by a numerical value, and the second element data being of the same type as the first element data, for each of the second sampling ranges. Second determining an average value; (e3) calculating a second relative difference for each of the second sampling ranges by dividing the second average value for each of the second sampling ranges by a sum of the second average values ​​for each of the second sampling ranges; (e 4 ) for each of the second sampling ranges Second Relative Differenceand (f) generating the reference code corresponding to the reference data by concatenating, as character string data, a numerical value of a predetermined number of digits from the most significant digits of the reference data, and (f) the third step is (f1) generating a reference code for each of the first sampling ranges obtained based on the reference code between the corresponding first sampling ranges and the corresponding second sampling ranges. The first relative difference or for each of the second sampling ranges obtained based on the numerical value of a predetermined number of digits from the uppermost position and the specific target code The second relative difference or (f2) determining that the reference code and the specific target code match if each of the differences falls within a predetermined tolerance range.

[0006] According to the above configuration, a new technique is provided for encoding target data such as images and audio with a small amount of data. [Brief description of the drawings]

[0007] [Figure 1] FIG. 1 is a diagram for explaining the encoding method according to the first embodiment. [Diagram 2] FIG. 2 is a diagram showing an example of the configuration of a database of codes generated by the above-mentioned method and contents corresponding to each code. [Diagram 3] FIG. 3 is a diagram conceptually explaining the flow of matching using a database. [Figure 4] FIG. 4 is a diagram for explaining a specific method for performing flexible matching. [Diagram 5] FIG. 5 is a line graph showing the differences corresponding to the respective sampling ranges S1 to S9 in the example shown in FIG. [Figure 6] FIG. 6 is a block diagram showing an example of the configuration of a code generating device for generating code using the above-mentioned method. [Figure 7]FIG. 7 is a block diagram showing an example of the configuration of a user device that performs matching of target data using a code generated by the above method. [Figure 8] FIG. 8 is a diagram showing an example of the configuration of a computer that realizes the information processing unit in each of the code generation device and the user device. [Figure 9] Fig. 9(A) is a flowchart showing the operation procedure of the code generating device, and Fig. 9(B) is a flowchart showing the operation procedure of the user device. [Figure 10] FIG. 10 is a diagram for explaining application example 1 of the encoding technique according to the above embodiment. [Figure 11] FIG. 11 is a diagram for explaining application example 2 of the encoding technique according to the above embodiment. [Figure 12] FIG. 12 is a diagram for explaining application example 3 of the encoding technique according to the above embodiment. [Figure 13] FIG. 13 is a diagram for explaining application example 4 of the encoding technique according to the above embodiment. [Figure 14] FIG. 14 is a diagram for explaining application example 5 of the encoding technique according to the above embodiment. [Figure 15] FIG. 15 is a diagram for explaining application example 6 of the encoding technique according to the above embodiment. [Figure 16] FIG. 16 is a diagram for explaining application example 7 of the encoding technique according to the above embodiment. [Figure 17] FIG. 17 is a diagram for explaining the encoding method according to the second embodiment. [Figure 18] FIG. 18 is a diagram showing an example of a code expressed in hexadecimal character code. [Figure 19] FIG. 19 is a diagram for explaining a matching method using a code according to the second embodiment. [Figure 20] FIG. 20 is a diagram for explaining the encoding method according to the third embodiment. [Figure 21] FIG. 21 is a diagram for explaining a matching method using a code according to the third embodiment. [Figure 22] FIG. 22 is a conceptual diagram showing a method of using each embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0008] First Embodiment 1 is a diagram for explaining the coding method of the first embodiment. Here, image data is taken as an example of data to be coded (target data) for explanation. In addition, coding in this specification refers to generating character string data that can identify the target data based on a predetermined numerical value included in the target data.

[0009] Assume that the target data is image data as shown in Fig. 1(A). The image data may be a color image or a monochrome image. The image data includes values ​​(element data) such as brightness, hue, saturation, and color difference for each pixel. For ease of understanding, the following explanation uses brightness values ​​as element data.

[0010] As shown in Fig. 1(B), the image data is divided into multiple sampling ranges. Here, the image data is divided into nine sampling ranges S1, S2, S3, ..., S9. Each sampling range S1, etc. has a width and height of 40 pixels, for example, and each sampling range S1, etc. contains a total of 1600 pixels. The brightness value (number of gradations) of each pixel is given as a numerical value in the range of 0 to 255.

[0011] In this case, if the pixel values ​​(brightness values ​​here) of the pixels included in each sampling range S1 etc. are px1, px2, ...px1600, the average pixel values ​​in each sampling range S1 etc. can be calculated as follows: Sum of pixel values: sumSx=px1+px2+px3+···px1600 Average pixel value: Ax=sumSx / 1600

[0012] 1C shows an example of the average values ​​of pixel values ​​in each sampling range S1 etc. In the example shown, the average value A1 in sampling range S1 is 128, the average value A2 in sampling range S2 is 86, the average value A3 in sampling range S3 is 134, the average value A4 in sampling range S4 is 101, the average value A5 in sampling range S5 is 68, the average value A6 in sampling range S1 is 101, the average value A7 in sampling range S7 is 115, the average value A8 in sampling range S8 is 91, and the average value A9 in sampling range S9 is 121.

[0013] Next, the relative difference in the average values ​​in each sampling range S1 etc. is calculated as follows. Sum of the average values: sumA=A1+A2+A3+···+A9 =945 Relative difference of the average values: Vx=(100×Ax) / 945

[0014] 1(D) shows an example of the relative difference of the average value in each sampling range S1 etc. In the example shown, the relative difference V1 in the sampling range S1 is 13.54%, the relative difference V2 in the sampling range S2 is 9.10%, the relative difference V3 in the sampling range S3 is 14.17%, the relative difference V4 in the sampling range S4 is 10.68%, the relative difference V5 in the sampling range S5 is 7.19%, the relative difference V6 in the sampling range S6 is 10.68%, the relative difference V7 in the sampling range S7 is 12.16%, the relative difference V8 in the sampling range S8 is 9.62%, and the relative difference V9 in the sampling range S9 is 12.80%.

[0015] Using the relative difference in each of these sampling ranges S1, etc., the numerical value from the most significant digit to the second decimal place of each relative difference (four digits in total) is extracted as character string data. For example, the character string data "1354" is extracted from sampling range S1. Note that if there is no value in the tens digit or in the decimal place, the value is set to 0. For example, 9.1% would be "0910".

[0016] In this manner, the character string data extracted from each sampling range S1 etc. is concatenated. There is no particular limitation on the concatenation order and it can be determined arbitrarily. Here, the sampling ranges are concatenated in the order of S1, S2, ... S9. As a result, in the above example, character string data "135409101417106807191068121609621280" is obtained (see FIG. 1(E)). In this embodiment, the character string data generated in this manner is used as a code corresponding to the target data.

[0017] The relative difference does not necessarily have to be expressed as a percentage. The order in which the character string data corresponding to each sampling range is concatenated can be any order, provided that a predetermined order is always used; for example, it can be decided that sampling ranges are concatenated in the order S9, S8, ... S1. It is also arbitrary to decide up to what order the relative difference is used in each sampling range. The number of divisions of the sampling range can also be arbitrary. Reducing the number of divisions of the sampling range enables simpler and faster matching, while increasing the number of divisions of the sampling range enables more accurate matching.

[0018] FIG. 2 is a diagram showing an example of a database configuration of codes generated by the above-mentioned method and contents corresponding to each code. Here, an image is taken as an example of contents. As shown in the figure, codes corresponding to each image 1, 2, etc. are generated in advance by the above-mentioned method, and a database linking them is constructed. For example, image 1 corresponds to code "2110...", and image 2 corresponds to code "1053...". Here, images 1, 2, etc. as contents may be image data itself, or may include additional information corresponding to the contents of the image data (for example, a character string representing an object included in the image, position information in real space, time information, and other arbitrary additional information). By constructing such a database in advance, it is possible to use each code in this database as reference data to check which content the specific target data corresponds to.

[0019] Fig. 3 is a diagram conceptually explaining the flow of matching using a database. For example, image data obtained by photographing is coded using the above method, and a specific target code ("8841..." in the illustrated example) that is a code corresponding to this image data is generated. This generated specific target code is compared with each reference code in the database, and if a reference code that matches the generated specific target code exists in the database, the content corresponding to the photographed image ("Image 37" in the illustrated example) can be identified.

[0020] For example, it is assumed that image data is obtained by photographing an arbitrary space using a terminal device (e.g., a smartphone) equipped with a camera. It is also assumed that an application program capable of executing the above-mentioned coding is pre-installed in the terminal device, and that a database such as that shown in FIG. 2 is stored in the terminal device. The database may be placed in a server capable of communicating with the terminal device via a network. In this case, for example, if content corresponding to an image of a landscape photographed by a user with a camera is identified, it is possible to realize a usage such that information related to the content is displayed on the screen of the user's terminal device.

[0021] According to the study by the inventors of the present application, if the target data is image data obtained by photographing a space, it is possible to perform matching and judgment with sufficient accuracy for practical purposes by encoding the data with a sampling range of 64 or more. The data volume of the code generated under these conditions is about 192 bytes, which can be said to be a very small data volume.

[0022] According to the principle of matching explained using Figure 3, if the character string data of the codes all match, the content corresponding to each code is determined to be the same. However, by setting the tolerance range for the judgment as described below, more flexible matching can be performed.

[0023] Fig. 4 is a diagram for explaining a specific method for performing flexible matching, and Fig. 5 is a line graph showing the differences corresponding to the respective sampling ranges S1 to S9 in the example shown in Fig. 4 above.

[0024] In the example shown in FIG. 4(A), the character string data of each of the reference code A and the identification target code B is divided into four digits corresponding to the above-mentioned sampling range, each character string data is treated as a four-digit numerical value, and the difference (AB) between the numerical values ​​is calculated for each corresponding sampling range. For example, the numerical value corresponding to the sampling range S1 is "1354" in the reference code and "1368" in the identification target code, so the difference between the two is "-14". Similarly, the differences between the other sampling ranges S2 to S9 are calculated as +15, 0, -12, +19, +17, +15, 0, and +18. In this case, if the allowable range of the difference corresponding to each sampling range is previously set to, for example, ±20, in the example shown in FIG. 4(A), the differences corresponding to all of the sampling ranges S1 to S9 fall within this allowable range. In such a case, it can be determined that the reference code and the identification target code are "identical (matching)". As shown in FIG. 5, when it can be determined that the reference code and the identification target code are identical (A and B), the graphs almost overlap. On the other hand, in the example shown in FIG. 4(B), the differences in the sampling ranges S1 to S9 for the character string data of the reference code A and the specific target code C are +74, -101, +27, 0, -261, +38, +126, -249, and +273. If the allowable range of the difference corresponding to each sampling range is previously set to ±20, in the example shown in FIG. 4(B), only the difference corresponding to the sampling range S4 falls within the allowable range, and the differences corresponding to the other sampling ranges do not fall within the allowable range. In such a case, it can be determined that the reference code and the specific target code are "different (mismatched)". In this case, as shown in FIG. 5, the graphs do not overlap when it can be determined that the reference code and the specific target code are different (A and C).

[0025] If no tolerance is set, it means that the image capture conditions (reflected light, ambient light, etc.) are exactly the same. In contrast, if a tolerance is set, it becomes possible to derive a "solution" that corresponds to slight environmental changes such as lighting conditions and the user's senses. By using this tolerance (environmental variable), it is possible to realize a judgment that is close to the sense of the human eye. The size of the tolerance can be set arbitrarily, but according to the inventor's investigation, it is preferable to set it to about ±20 as mentioned above.

[0026] 6 is a block diagram showing an example of the configuration of a code generation device for generating code using the above-mentioned method. The illustrated code generation device 1 includes an information processing unit 10, a camera 11, a microphone 12, a storage device 13, an operation device 14, and a display device 15.

[0027] The information processing unit 10 performs information processing related to code generation. This information processing unit 10 includes, as functional blocks, a sampling processing unit 20, a feature value calculation unit 21, an encoding processing unit 23, and a code recording processing unit 24. The information processing unit 10 is realized by using a computer having, for example, a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), etc., and executing a predetermined operation program on this computer. In this embodiment, the sampling processing unit 20 corresponds to the "division unit", the feature value calculation unit corresponds to the "average value calculation unit" and the "relative difference calculation unit", and the encoding processing unit 23 corresponds to the "code generation unit".

[0028] The camera 11 captures an image of a space, an object, etc. that can be a target for code generation, and generates an image of the captured image. The image referred to here is not necessarily limited to an image in the visible light range, but may be, for example, a thermal image or an infrared image. The camera 11 outputs image data (or image signals) of the captured image to the information processing unit 10.

[0029] The microphone 12 picks up sounds that can be used for code generation and converts them into electrical signals. The sounds referred to here are not limited to sounds in the audible range, but may be ultrasonic, low-frequency sounds, etc. The sound signals output from the microphone 12 are output to the information processing unit 10, where they are converted into digital data and then captured.

[0030] The storage device 13 stores programs and various data necessary for the configuration and operation of the information processing unit 10, and also stores a database of codes generated by the information processing unit 10. The storage device 13 is a non-volatile storage device such as a hard disk drive (HDD) or a solid state drive (SSD).

[0031] The operation device 14 is used for inputting information necessary for the operation of the information processing unit 10, and may include, for example, a keyboard, a mouse, a switch, a touch panel, etc. The display device 15 is used for displaying information related to the operation of the information processing unit 10, and may include, for example, a liquid crystal display device, an organic EL display device, etc.

[0032] The sampling processing unit 20 performs processing to divide the target data for code generation into a plurality of sampling ranges (see FIG. 1(B)). The target data is, for example, image data or sound data. In the case of sound data, it is possible to divide the data into predetermined time intervals, and the sound data included in these intervals can be treated as data corresponding to each sampling range (see the third embodiment described later).

[0033] The feature value calculation unit 21 calculates the average value of the predetermined data included in each sampling range divided by the sampling processing unit 20 (see FIG. 1(C)), and calculates the relative difference of the average values ​​corresponding to each sampling range (see FIG. 1(D)). The relative difference is expressed as a percentage, for example, as in the above example, but is not limited to this.

[0034] The encoding unit 23 uses the relative difference (or average value) corresponding to each sampling range calculated by the characteristic value calculation unit 21 to generate a code (a code obtained by concatenating character string data) corresponding to the target data.

[0035] The code recording processing unit 24 associates the code generated by the encoding processing unit 23 with information data identifying the content corresponding to this code and records them in the storage device 13, thereby generating a code database (see FIG. 2).

[0036] 7 is a block diagram showing an example of the configuration of a user device that performs matching of target data using a code generated by the above method. The illustrated user device 2 includes an information processing unit 50, a camera 51, a microphone 52, a storage device 53, an operation device 54, and a display device 55. This user device 2 is realized by, for example, installing a predetermined application program in advance on an information processing device such as a smartphone owned by a user.

[0037] The information processing unit 50 performs information processing related to code matching. This information processing unit 50 includes, as functional blocks, a sampling processing unit 60, a feature value calculation unit 61, an encoding processing unit 63, and a matching processing unit 64. The information processing unit 50 is realized by using a computer having, for example, a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), etc., and causing the computer to execute a predetermined operation program.

[0038] The configurations and operations of the camera 51, microphone 52, storage device 53, operation device 54, and display device 55 are similar to those of the camera 11, microphone 12, storage device 13, operation device 14, and display device 15 in the above-described code generation device 1, so detailed descriptions thereof will be omitted here. The configurations and operations of the sampling processing unit 60, feature value calculation unit 61, and encoding processing unit 63 are also similar to those of the sampling processing unit 20, feature value calculation unit 21, and encoding processing unit 23 in the above-described code generation device 1, so detailed descriptions thereof will be omitted here.

[0039] The matching processing unit 64 of the information processing unit 50 matches the identified target code obtained by the processing of the sampling processing unit 60, the feature value calculation unit 61, and the encoding processing unit 63 with a database (see FIG. 2) pre-stored in the storage device 53, and identifies the content corresponding to the identified target code. The specific matching method used by the matching processing unit 64 may be a method that requires a perfect match between the codes, or may be a flexible method that sets an allowable range for the difference between the codes as described above. The identification result is displayed on the display device 55, for example.

[0040] FIG. 8 is a diagram showing an example of the configuration of a computer that realizes the information processing unit in each of the code generation device and the user device. The computer shown in the figure includes a CPU 201, a ROM 202, a RAM 203, a storage device 204, and an external interface (I / F) 205 that are communicably connected to each other. The CPU 201 operates based on a basic control program read from the ROM 202, and realizes the function of the controller 13 described above by reading and executing a program (application program) 206 stored in the storage device 204. The RAM 203 temporarily stores data to be used during the operation of the CPU 201. The storage device 204 is a non-volatile storage device such as a hard disk or a solid state drive, and stores various data such as the program 206. The external interface 205 is an interface that connects the CPU 201 to an external device, and is used to connect the CPU 201 to, for example, a camera 11 (51), a microphone 12 (52), an operation device 14 (54), and a display device 15 (55).

[0041] Fig. 9(A) is a flowchart showing the operation procedure of the code generation device. Fig. 9(B) is a flowchart showing the operation procedure of the user device. In either operation procedure, the order of processing can be changed as appropriate as long as no contradiction or inconsistency occurs in the results of information processing. In addition, other processing not mentioned here can be added, and such implementations are not excluded.

[0042] First, the operation procedure of the code generating device will be described. When target data is input to the code generation device 1 (step S11), the sampling processing unit 20 performs a process of dividing the target data into a plurality of sampling ranges (step S12). Here, the target data may be image data input from the camera 11, sound data corresponding to a signal input from the microphone 12, or image data or sound data previously stored in the storage device 13. The target data may also be data transmitted from a server or the like (not shown) via a network and received by the code generation device 1.

[0043] Next, the feature value calculation unit 21 calculates the average value of predetermined element data included in each sampling range divided by the sampling processing unit 20 (step S13). The element data here may be data that is included in the target data as described above and that forms the contents, features, or characteristics of the target data. For example, as described above, in the case of image data, it may be the brightness of each pixel, and in the case of sound data, it may be the volume of sound at each predetermined sampling time.

[0044] Next, the characteristic value calculation unit 21 obtains the relative difference of the average values ​​corresponding to each sampling range (step S14). The relative difference is expressed as a percentage, for example, as in the above example, but is not limited to this.

[0045] Next, the encoding unit 23 generates a reference code (a code obtained by concatenating character string data) corresponding to the target data, using the relative difference corresponding to each sampling range calculated by the feature value calculation unit 21 (step S15).

[0046] Next, the code recording processing unit 24 associates the reference code generated by the encoding processing unit 23 with information data identifying the content corresponding to this reference code and records them in the storage device 13, thereby generating a database of reference codes (see Figure 2) (step S16).

[0047] By repeating the above series of information processing as many times as necessary, reference codes corresponding to each of the multiple contents are generated, and a database containing the information is obtained. The generated database may be transmitted to and stored in a server device (not shown) connected to the network so as to be able to communicate with the content. This allows the database to be shared by multiple users via the network.

[0048] Next, the operation procedure of the user equipment will be described. When specific target data is input to the user device 2 (step S21), the sampling processing unit 60 performs processing to divide the specific target data into a plurality of sampling ranges (step S22). The specific target data here may be image data input from the camera 11, sound data corresponding to a signal input from the microphone 12, or image data or sound data previously stored in the storage device 13.

[0049] Next, the feature value calculation unit 61 calculates the average value of predetermined element data included in each sampling range divided by the sampling processing unit 60 (step S23). The element data here may also be data that is included in the target data as described above and is used to form the contents, features, or characteristics of the target data.

[0050] Next, the characteristic value calculation unit 61 calculates the relative difference of the average values ​​corresponding to each sampling range (step S24).

[0051] Next, the encoding processing unit 63 generates a specific target code (a code obtained by concatenating character string data) corresponding to the specific target data, using the relative difference corresponding to each sampling range calculated by the feature value calculation unit 61 (step S25).

[0052] Next, the matching processing unit 64 refers to a database stored in advance in the storage device 53 and identifies the content corresponding to the identification target code generated in step S25 (step S26). The identification result is displayed on the display device 55 as appropriate. The database stored in the storage device 53 is, for example, transmitted from a server or the like (not shown) via a network and received by the user device 2.

[0053] The above is the basic content regarding the code generation and matching of the first embodiment, and next, some application examples will be described. Note that the code generation and matching according to the following application examples are all executed by the above-mentioned code generation device 1 and user device 2 (or other devices).

[0054] FIG. 10 is a diagram for explaining an application example 1 of the encoding technology according to the embodiment described above. Here, an application example to face recognition technology is shown. The face recognition technology here refers to a technology for identifying a person corresponding to a face included in a certain image. The certain image may be a still image or a moving image captured by a camera. In the case of a moving image, a still image obtained for each frame can be used. As shown in FIG. 10(A), it is assumed that face image data 300 of a certain person from the front is given. For this face image data 300, face image data corresponding to a bird's-eye view of the person from each of the up, down, left, and right directions is automatically generated using a known technology. Of course, the face image data may be directly captured from the person, instead of being automatically generated. Here, a total of 15 face image data are shown, including the face image data 300 from the front and face image data corresponding to each of the up, down, left, and right directions that are automatically generated (or captured) around the face image data.

[0055] The face image data are coded by the code generating device 1 as shown in FIG. 10(B) to obtain reference codes. Here, code 1 corresponding to the face image data 300 of the front, and codes 2 to 15 corresponding to the other face image data in the up, down, left and right directions are obtained. These 15 codes 1 to 15 are linked as a set of face recognition data. For example, identification information such as "xxxxx_CEC" can be given to each code as identification information (FaceID) and linked to them. In addition, attribute information (such as name, age, affiliation, etc.) related to the person to be identified is linked to the identification information.

[0056] By generating and storing such face authentication data in advance, for example, in the user device 2 (or other device), face authentication can be performed based on image data obtained from a camera in real time. There are various specific collation methods, and for example, if any one of Code 1 to Code 15 matches, the person may be identified, or a condition may be that multiple codes match. The code obtained by the encoding technology of this embodiment is irreversible character string data (text data), and face image data itself is not used. Therefore, for example, when face authentication data is stored on a network server, privacy can be protected even if information is leaked. In addition, since the data size of one code is several hundred bytes, the data capacity required for face authentication can be dramatically reduced. For example, if the data size of one face image data is 100 kilobytes, the data capacity of 15 face image data is 1500 kilobytes, whereas the data size of one code can be about 192 bytes, for example, so that the data capacity of 15 codes is 2880 bytes, and the difference in data capacity is obvious.

[0057] FIG. 11 is a diagram for explaining an application example 2 of the coding technique according to the embodiment described above. Here, an application example is shown in which image data such as space is divided into a plurality of partial images, and a code is generated for each partial image to perform image recognition (spatial recognition). Specifically, the code generating device 1 described above divides image data 400 as target data into a plurality (here, 12) partial images A to L, as shown in FIG. 11(A). Then, the above coding is performed for each of the partial images A to L. The code generated corresponding to the partial image A is represented as A_code. The same applies to the codes corresponding to the other partial images. The 12 codes such as A_code generated in this way are linked and recorded as one reference code group.

[0058] Similarly, the image data as the specified target data is divided into a plurality of partial images by the user device 2 (or another device) and a code corresponding to each partial image is generated. For example, a code is generated for a still image captured in real time by the user's camera or for one frame of a moving image. At this time, for example, as shown in FIG. 11(B), it is assumed that the shooting range of the image data 400 corresponding to the reference data group and the image data 410 as the specified target data by the user partially overlap. Here, it is assumed that the portions corresponding to the partial images E, F, I, and J overlap.

[0059] In this state, if the code group generated from the image data 410, which is the identification target data, is referenced to the reference code group in a brute force manner, the number of matching partial images T=4 and the number of mismatching partial images F=8 for the total number of partial images N=12 (see FIG. 11(C)). If the matching rate (recognition rate) is defined as T / N and the mismatch rate is defined as F / N, in this example, the matching rate is about 33% (T / N=4 / 12) and the mismatch rate is about 66% (F / N=8 / 12). The percentage of the matching rate or higher that is required to determine that the identification target data and the reference data match varies depending on the application, and can be set appropriately. The matching rate can also be used to determine similarity. For example, if the matching rate is about 33% as described above, it can be determined that there is a high possibility that they are similar, or that they are taken in a close location. Using such a method, any frame-in can be handled. Also, the greater the total number N of partial images, the more the recognition accuracy can be improved. Even in this case, the search, collation, and judgment are basically performed using text data, so high-speed processing equivalent to that of commonly known text searches can be performed.

[0060] FIG. 12 is a diagram for explaining an application example 3 of the coding technique according to the embodiment described above. Here, an application example in which the up-down direction of an object is determined is shown. The object may be of any shape asymmetric in the up-down direction. For example, the up-down direction may be determined in order to crack pistachios with an industrial machine.

[0061] As shown in Fig. 12(A), an object 500 is divided into two parts a and b by focusing on its asymmetry in the vertical direction. As an example, part a is regarded as a triangle, and part b is regarded as a trapezoid. The base of part a is equal to the lower base of part b, and the heights of both are also equal. When parts a and b are fitted into regions 510 and 512 of the same size (rectangular regions indicated by dotted lines in the figure) and the remaining parts other than parts a and b are compared, the remaining part of rectangular region 510 corresponding to part a is larger than that of rectangular region 512 corresponding to part b. Based on such knowledge, coding technology is applied.

[0062] Specifically, as shown on the left side of FIG. 12(B), first, an image of a base area is captured in the absence of the object 500, and the image data 520 is divided into multiple partial images (for example, several hundred), and a reference code corresponding to each partial image is generated, and the reference codes are recorded.

[0063] Next, as shown in the center of Fig. 12(B), the edges of the object 500 are detected based on image data 522 obtained by photographing the base area with the object 500 placed therein, and an effective area 524 (a rectangular area indicated by a thick line in the figure) in which the object 500 exists is extracted. As shown in Fig. 12(B), the inside of this extracted rectangular area 524 is divided so that the upper and lower halves each have the same number of partial images, and each partial image is coded. The partial images are divided in the same way as when the reference code group was generated.

[0064] Next, when the reference code group is compared with the specific object code group generated in a state where the object 500 is placed, the codes corresponding to the area where the object exists do not match, but the codes corresponding to the other areas match. In FIG. 12(B), the areas where the codes match are shown with diagonal lines. As shown in FIG. 12(C), the number of areas where the codes match is counted separately for the upper half area 524a of the effective area and the lower half area 524b of the effective area. In the illustrated example, 24 codes match in the upper half area 524a of the effective area, and 13 codes match in the lower half area 524b of the effective area. In this case, the upper half area 524a of the effective area corresponds to the above-mentioned part a, so it can be determined as "upper", and the lower half area 524b of the effective area corresponds to the above-mentioned part b, so it can be determined as "lower".

[0065] FIG. 13 is a diagram for explaining an application example 4 of the coding technique according to the embodiment described above. Here, an application example is shown in which an object (an animal, as an example) in a video (image) is coded in advance, and the object present in the video is recognized using the reference code group. Specifically, first, as shown in FIG. 13(A), edge detection is performed on an object 600 (an animal in this case) in the video using a known technique, and an edge 610 of the object is extracted as shown in FIG. 13(B). Next, as shown in FIG. 13(C), a plurality of rectangular areas are mapped on the extracted edge 610 and its internal area 612. The number of rectangular areas can be set arbitrarily, and the greater the number, the more the recognition accuracy can be improved. Then, for the partial images of each mapped rectangular area, a reference code corresponding to each partial image is generated by the above-mentioned method, and the reference code group is recorded. In the same manner, a reference code group corresponding to each shooting direction of the object is generated and recorded. Information about the object (for example, attribute information such as the name of an animal) is linked to these reference code groups and recorded.

[0066] Next, for example, in a video captured in real time, the edges of the object are detected and rectangular regions are mapped in the same manner as described above, and the partial images of each rectangular region are coded to generate a specific object code group corresponding to the object in the video. By comparing this specific object code group with a reference code group recorded in advance, the object contained in the video can be recognized and its presence or absence and attributes such as its name can be identified.

[0067] In this way, by combining the existing edge detection technology with the coding technology of this embodiment, it is possible to register and verify the code immediately, improving the recognition accuracy. The same applies when the target object is space or electronic data. The coding technology of this embodiment can realize the processes from generation to registration, comparison, verification, and judgment in real time, improving the recognition accuracy and speed.

[0068] FIG. 14 is a diagram for explaining an application example 5 of the coding technique according to the embodiment described above. Here, for example, an application example to an appearance inspection process during the manufacture of industrial products is shown. Specifically, first, as shown in FIG. 14(A), the appearance of a product to be inspected is photographed to obtain image data 700. Next, as shown in FIG. 14(B), the image data 700 is rotated 360° at any angle (for example, 1°) to obtain image data 701, 702, etc. at each angle. Then, coding is performed based on the image data at each angle, and each code is recorded as a reference code. Next, as shown in FIG. 14(C), in an actual appearance inspection process, for example, codes are generated from image data 710-713 obtained by photographing each product moving on a belt, and these codes are compared with the reference code recorded in advance. As a result, even if the angle at which the product is arranged is not constant, if the shape and color tone are the same, it can be determined to be an acceptable product (OK product), and if there is a partial chip or a difference in color tone, it can be determined to be a defective product (NG product). In the illustrated example, the products corresponding to the image data 711 and 713 are acceptable products, while the products corresponding to the image data 710 and 713 are defective products.

[0069] In this way, when generating the reference code in advance, if one good product (a product without defects) is photographed in one arrangement (angle), multiple reference codes that can be identified even if the direction or angle changes can be automatically generated by in-memory processing. Also, by embedding color information in the code, it becomes possible to judge not only by shape but also by color tone.

[0070] Fig. 15 is a diagram for explaining an application example 6 of the coding technique according to the embodiment described above. For example, if it is confirmed that products flow in a fixed arrangement in a visual inspection process, it is not necessary to generate reference codes with different angles. In this case, a single reference code corresponding to a non-defective product in a fixed arrangement can be used. Here, as an example, a case of detecting defects such as missing parts, printing problems, and scratches on a computer keyboard is considered.

[0071] First, as shown in FIG. 15(A), the appearance of the product to be inspected is photographed to obtain image data 800, which is then coded and recorded as a reference code. At this time, as shown in FIG. 15(B), it is preferable to divide the image data 800 of the target product into a plurality of rectangular regions to obtain partial images 801 corresponding to each region, and obtain a reference code group, which is a set of reference codes corresponding to each partial image 801. Next, in the actual appearance inspection process, for example, a specific target code (group) is generated from the image data obtained by photographing each product moving on a belt, and this is compared with the reference code (group) recorded in advance. This makes it possible to detect defects such as missing keyboards, printing problems, scratches, etc., if any. In this case, as shown in FIG. 15(C), it is also possible to identify the region 810 where the defect exists based on the mismatched reference code.

[0072] FIG. 16 is a diagram for explaining an application example 7 of the coding technique according to the embodiment described above. Here, a method for obtaining the shape and area of ​​an object will be explained. The object is assumed to be placed in a certain shooting area and photographed. In this case, image data of the shooting area is first obtained when the object is not placed, and then this is divided into a plurality of areas, and partial images of each area are coded. This results in a reference code group consisting of a plurality of codes Cd11, Cd12, ..., Cd99 (81 in the illustrated example) as shown in FIG. 16(A), and this reference code group is recorded.

[0073] Next, image data of the photographed area where the object is placed is obtained, divided into a plurality of areas, and partial images of each area are coded. As a result, a group of specific object codes is obtained. These are compared with the above-mentioned group of reference codes. Then, as shown in FIG. 16(B), a region 901 where the codes match (a rectangular region with no pattern in the figure) and regions 902 and 903 where the codes do not match (rectangular regions with patterns in the figure) are obtained. The regions 902 and 903 where the codes do not match are regions where the object exists. Here, the matching rate is divided in 10% increments. Among the regions where the codes do not match, for example, the region 902 with a matching rate of 30% to 40% in the illustrated example is connected to obtain the edge of the object, that is, the shape. In addition, the area within this edge can be obtained to obtain the area of ​​the object in a planar view. In addition, the unevenness of the object can be estimated based on the part of the region 903 with a different matching rate within the edge (here, for example, a matching rate of 60% to 70%). There are various ways to calculate the match rate between codes, but a simple example is to compare the codes in order, character by character, count the number of matching characters, and divide this number by the total number of characters. For example, if a code contains 384 characters and 370 of them match, the match rate is about 96%.

[0074] <Second embodiment> FIG. 17 is a diagram for explaining the coding method of the second embodiment. Here, image data is taken as an example of data to be coded (target data) for explanation. Note that the configurations of the code generation device and user device are similar to those of the first embodiment (see FIG. 6 and FIG. 7), and only the information processing method relating to code generation and matching is different. Here, explanation of the device configuration is omitted, and the information processing method for code generation and matching will be described in detail.

[0075] As shown in FIG. 17A, image data as target data (reference data) is divided into a plurality of sampling ranges. In the illustrated example, the image data is divided into 64 sampling ranges S11, S12, S13, S88 arranged in 8 rows and 8 columns, but the number of divisions is not limited to this. The total number of pixels included in each of the sampling ranges S11 to S88 is, for example, 1000 to 2000. Each pixel contains color information in YUV format based on a known format. Here, Y represents the luminance value, U (Cb) represents the difference between the luminance value and the blue component, and V (Cr) represents the difference between the luminance value and the red component. Hereinafter, the respective values ​​are referred to as Y value, U value, and V value. This process is performed by the sampling processing units 20 and 60.

[0076] At this time, the average values ​​of the Y, U and V values ​​are calculated for each of the sampling ranges S11 to S88. The method of calculating the average values ​​is the same as in the first embodiment described above. For example, for the sampling range S11, the average values ​​are calculated by calculating the sum of the Y, U and V values ​​of all pixels contained in this sampling range S11 and dividing the sum by the total number of pixels in the sampling range S11. The same applies to the other sampling ranges. This process is executed by the feature value calculation units 21 and 61.

[0077] 17B shows an example of the average values ​​of the Y, U, and V values ​​in each sampling range. Here, the average values ​​of the Y, U, and V values ​​in each sampling range are represented by Y11, the average value of the U values, and V11 in sampling range S11. For example, in sampling range S12, the average values ​​are Y12=119, U12=128, and V12=142.

[0078] A code is generated using the average values ​​of the Y, U, and V values ​​of each sampling range obtained as described above. As an example, as shown in FIG. 17C, the average values ​​are linked in the order of Y, U, and V values ​​as follows: Y11, Y12, Y13, . . . Y21, Y22, . . . Y87, Y88, U11, U12, U13, . . . U21, U22, . . . U87, U88, V11, V12, V13, . . . V21, V22, . . . Y87, Y88. Note that this is only an example, and the order of linking the Y, U, and V values ​​can be set appropriately. In this way, a reference code corresponding to the image data as the target data (reference data) is generated. This process is executed by the encoding processing units 23 and 63.

[0079] If the values ​​of Y11 to Y88, U11 to U88, and V11 to V88 are each expressed as a hexadecimal character string, for example, then the data size of the code obtained when a sampling range of 64 divisions with 8 rows and 8 columns is set as described above will be 192 bytes (=64×3). This corresponds to 384 characters of text data. An example of a code expressed in hexadecimal character code is shown in Figure 18. Since the data amount per character in hexadecimal characters is 4 bits, the number of characters is 384, but the amount of data when recorded will be 192 bytes.

[0080] According to the study by the inventors of the present application, this 192-byte data amount is the practical minimum value for performing comparison, collation, judgment, and recognition in real time without complex calculations. In other words, 192 bytes is the minimum value of an effective code that can achieve a recognition rate of about 98% for visualized target data such as space, landscape, object, heat source video, video, and images. This data size does not depend on the area or data size of the original video or image.

[0081] For example, consider the amount of data generated when 24 codes are generated per second to correspond to a video with a frame rate of 24 fps. If the data size of a code corresponding to one image is 192 bytes as shown above, the number of codes corresponding to a 30-minute video will be 43,200, with the amount of data being approximately 7.9 MB. Similarly, the number of codes corresponding to a 1-hour video will be 86,400, with the amount of data being approximately 15.8 MB, the number of codes corresponding to a 2-hour video will be 172,800, with the amount of data being approximately 31.6 MB, and the number of codes corresponding to a 1-day video will be 2,073,600, with the amount of data being approximately 379.7 MB. Even if each code is accompanied by associated information such as coordinates (longitude, latitude, altitude: 24 bytes), time (date and hour, minute, second: 8 bytes), and a comment of 100 double-byte characters (200 bytes), the amount of data corresponding to a 30-minute video is approximately 17.5MB, that corresponding to a 1-hour video is approximately 34.9MB, that corresponding to a 2-hour video is approximately 69.9MB, and that corresponding to a 1-day video is approximately 838.5MB. If one code is generated for one frame per second, the amount of data will be 1 / 24 of the above amount. In this case, the amount of data for a 1-day video will be extremely small, at 15.8MB (without associated information) or 34.9MB (with associated information).

[0082] In addition, since this code contains color information for each of the YUV colors, this color information can be used to obtain color information in RGB format using the following conversion formula. R = (1.164 x (Y value - 16)) + (1.596 x (V value - 128)) G = (1.164 x (Y value - 16)) - (0.391 x (U value - 128)) - (0.813 x (V value - 128)) B = (1.164 x (Y value - 16)) + (2.018 x (U value - 128))

[0083] FIG. 19 is a diagram for explaining the matching method using the code of the second embodiment. Here, in order to simplify the explanation, a case where a sampling range of 3 rows and 3 columns is set is considered. In the figure, the left side shows a reference code that has been generated and recorded in advance, and the right side shows a specific target code obtained from the specific target data. Note that the actual form of each code is a character string as shown in FIG. 17(C) above, but here it is expressed so that the correspondence for each sampling range is easy to understand. As the matching method, for example, a simple comparison method and a relative comparison method can be considered. These processes are executed by the matching processing unit 64.

[0084] First, a simple comparison method will be described. For each sampling range, the differences between the Y value, U value, and V value of the reference code and the specific code are calculated, and the differences are squared. For example, in the illustrated example, the square of the difference in the Y value in the sampling range of row 1, column 1 is (202-195). 2 = 49. Similarly, the squared difference of the Y values ​​in the sampling range of 1 row and 2 columns is (195-212). 2 = 289. Similarly, the squared difference of the Y values ​​in a sampling range of 3 rows and 3 columns is (149-190). 2 =1681. The squares of the Y value differences are calculated in the same way for the other sampling ranges, and the sum of these is calculated. In the example shown, the sum of the squares of the Y value differences in each sampling range is 15114 (=49 + 289 + ··· +1681). Similarly, the squares of the differences between each value are calculated for the U value and V value, and the sums are 8694 and 3072, respectively.

[0085] Once the totals corresponding to the Y, U, and V values ​​are obtained in this way, they are compared with a preset reference value, and if each value is below the reference value, it is determined that the reference code and the specific target code match. The reference value can be set taking into consideration the application and the level of similarity required to "match," and can be set to, for example, a reference value equivalent to a match rate of 97% or more.

[0086] The reason why the differences between the Y, U, and V values ​​are squared is to make it easier to detect the difference between the reference code and the specific target code. Depending on the application, squaring may not be necessary, or conversely, the exponent may be increased further, such as by cubes.

[0087] Next, a method for relative comparison will be described. First, for the reference code, the square of the difference in Y value between the sampling range of row 1, column 1 and the sampling range of row 1, column 2 is calculated as Ya12. In the illustrated example, Ya12=(202-195) 2 = 49. Similarly, the squares of the differences in Y values ​​between the sampling range of row 1, column 1 and the eight sampling ranges thereafter are calculated as Ya13, Ya14,..., Ya19.

[0088] Next, the square of the difference in Y value between the sampling range of row 1, column 2 and the sampling range of row 1, column 3 is calculated as Ya23. In the illustrated example, Ya23=(195-213) 2 = 324. Similarly, the values ​​Ya24, Ya25, ... Ya29 are calculated by squaring the differences in Y values ​​between the sampling range of row 1, column 2 and the seven sampling ranges thereafter. Similarly, the values ​​Y34, Y35, ... Y49 are calculated by squaring the differences in Y values ​​between the sampling range of row 1, column 3 and the subsequent sampling ranges.

[0089] Similarly, the squared difference in Y values ​​is calculated between the sampling range of row 2, column 1 and the sampling range thereafter, the sampling range of row 2, column 2 and the sampling range thereafter, the sampling range of row 2, column 3 and the sampling range thereafter, the sampling range of row 3, column 1 and the sampling range thereafter, and the sampling range of row 3, column 2 and the sampling range thereafter.

[0090] Then, the squared differences in the calculated Y values ​​are all summed up. In the example shown, the total is 47502. Similarly, when the U and V values ​​are calculated, the total for the U values ​​is 73256, and the total for the V values ​​is 0.

[0091] Next, when the specific target codes are calculated in the same manner as above, the total value for the Y values ​​is 70,898, the total value for the V values ​​is 58,104, and the total value for the U values ​​is 18,272.

[0092] Then, the differences in the total values ​​of YUV between the reference code and the specific target code are calculated as ΔY=23396, ΔU=15152, and ΔV=18272.

[0093] When the differences in the total values ​​corresponding to the Y, U, and V values ​​are obtained in this way, they are compared with a preset reference value, and if each value is below the reference value, it is determined that the reference code and the specific target code match. The reference value can be set taking into consideration the application and the level of similarity required to "match," and can be set to, for example, a reference value equivalent to a match rate of 97% or more.

[0094] As with the simple comparison, the differences for the Y, U, and V values ​​are further squared to make it easier to detect the difference between the reference code and the specific target code. Depending on the application, squaring may not be necessary, or conversely, a larger exponent such as cubed may be used. Also, in the case of relative comparison, the difference between a certain sampling range and the subsequent sampling range was found in the above example, but the difference may be found by comparing each sampling range with all other sampling ranges in a brute force manner.

[0095] The code according to the second embodiment as described above can be applied to the various application examples described in the first embodiment. That is, although the code generation method differs between the first and second embodiments, each application example relates to a method of utilizing the generated code and is not affected by the code generation method.

[0096] <Third embodiment> FIG. 20 is a diagram for explaining the encoding method of the third embodiment. Here, sound data is taken as an example of data to be encoded (target data) for explanation. Note that the configurations of the code generation device and user device are similar to those of the first embodiment (see FIG. 6 and FIG. 7), and only the information processing method relating to code generation and matching is different. Here, explanation of the device configuration is omitted, and the information processing method for code generation and matching will be described in detail.

[0097] Generally, the three elements of sound are loudness, pitch, and timbre. Sound data coding is not a technique for reproducing the sound itself, but rather for the purpose of recognizing and identifying the sound by analyzing the sound data in real time, for example. In this embodiment, coding is performed with a focus on pitch among the three elements of sound.

[0098] FIG. 20(A) is an example of a waveform diagram obtained by performing a discrete Fourier transform on a sound collected by a microphone to obtain a time change in frequency. Note that, if necessary, a fundamental frequency may be calculated by cepstrum analysis. For such a waveform diagram, for example, a sampling interval Q is set to 0.2 seconds, and a frequency is extracted at each sampling interval. The extracted frequency value is then coded and recorded. At this time, related data such as the sampling interval Q, the sampling time (2.0 seconds in the illustrated example), and the title of the song are added and recorded. The sound data to be coded may be obtained in real time by collecting sound with a microphone, or may be data that has been collected and stored in advance. If the stored sound data is in a compressed format such as mp3, it is appropriately converted into an uncompressed format such as WAVE format and coded.

[0099] In the above information processing, data division for each sampling interval is performed by the sampling processing unit 20, frequency calculation is performed by the feature value calculation unit 21, coding is performed by the encoding processing unit 23, and the obtained code and related information are recorded by the code recording processing unit 24.

[0100] FIG. 20B is a diagram showing an example of coding. It shows an example of coding performed every 0.2 seconds based on the waveform of the sound data shown in FIG. 20A. For example, when the acquisition timing is 0.2 seconds, the frequency is extracted as 1046.50 Hz, and the corresponding code (indicated as "CEC" in the figure) is obtained as 0416. This note corresponds to "C6" in the scale name and 64 in the keyboard symbol (in the case of an 88-key keyboard). The same applies to other acquisition timings. In this example, the audible range is assumed to be 20 Hz to 20,000 Hz when the frequency is extracted, and the decimal points are truncated when coding. For example, in the case of 20.2 Hz, the value after truncation is "20", which is coded as "14" in hexadecimal. Similarly, in the case of 261.62 Hz, the value after truncation is "261", which is coded as "105" in hexadecimal. Similarly, in the case of 20000.0 Hz, the value after truncation is "20000", which becomes "4e20" when coded in hexadecimal. Therefore, the code generated in this embodiment can be expressed in a maximum of four hexadecimal characters, so the data size is a maximum of 2 bytes.

[0101] FIG. 21 is a diagram for explaining a matching method using a chord according to the third embodiment. As shown in FIG. 21(A), a chord corresponding to the pitch of a part (2000 ms) of a certain piece of music (Twilight Message) is obtained in advance by setting a sampling interval Q to 0.2 seconds, and is recorded as a reference chord group. In the figure, for the sake of simplicity, the chords are represented by the alphabets A to G, and it is assumed that the contents of the chords represented by the same alphabet match. In response to this, the identification target data of each piece of music given as real time (or sound data) is coded, and this is matched with the reference data group. In the example shown in the figure, since there is no part in each of the pieces of music (1) and (3) where the sequence of chords matches the reference data group, it can be determined that the piece is not a piece of music identified by the reference data group. On the other hand, since there is a part in the piece of music (2) where the sequence of chords matches the reference data group, it can be determined that the piece of music (2) is a piece of music (Twilight Message) corresponding to the reference data group. Regarding the match / mismatch between the codes at each sampling interval Q, the codes may be judged as "matching" if they are exactly the same, or may be judged as "matching" if they are within a certain error range (for example, an error range equivalent to ±10 Hz).

[0102] In the above information processing, data division for each sampling interval is performed by the sampling processing unit 60, frequency calculation is performed by the feature value calculation unit 61, coding is performed by the encoding processing unit 63, and the obtained code is matched with a reference data group by the matching processing unit 64.

[0103] Here, when encoding sound data, it is more preferable to set the sampling interval in consideration of the tempo, especially when the target is sound data of a musical piece. For example, in the case of a musical piece with a tempo (BPM) of 120, one minute contains 120 quarter notes (beats). In this case, there are two beats per second, and the interval between one beat is 0.5 seconds. In this case, by setting (adjusting) the sampling interval according to the tempo of the sound data, encoding corresponding to the timing of the notes is possible. Specifically, the sampling interval is Q, the tempo is B, and the length of the beat of the target note is X. Here, the length of the beat of the note is set, for example, as X=4 for a quarter note and X=8 for an eighth note. In this case, as shown in FIG. 21B, the sampling interval Q can be expressed as Q=(1 / ((B / 60)×(X / 4)))×1000. By setting the sampling interval Q using this relational expression and encoding the sound data, the accuracy of determining the similarity of the musical pieces is further improved. Specific values ​​of preferable sampling intervals Q are shown in the lower column of FIG.

[0104] By encoding sound data in this way, it is possible to realize control such as displaying the title of a song playing in the air on the user device 2, or displaying AR content (augmented reality content), videos, and still images that match the song on the user device 2. In addition, regarding the so-called problem of plagiarism of music, it is possible to present the similarity or non-similarity as an objective judgment, rather than a subjective judgment by humans. Furthermore, assuming that the heart sounds of living things are generally all different, even the heart sounds of twins are different, encoding the heart sounds makes it possible to perform person authentication based on the heart sounds (heart sound authentication). For example, it is possible to create unprecedented use cases, such as person authentication at gates at airports, etc.

[0105] According to the above-described embodiments, a novel technique is provided for encoding target data such as images and audio with a small amount of data, thereby enabling matching between specific target data and reference data with a low load and in a short processing time.

[0106] That is, according to each embodiment, for example, an image captured by a camera is divided into an arbitrary number of parts, and a unique code is generated and recorded for each of the divided blocks (sampling ranges) from information such as light source, brightness, and color, and this code can be used to perform high-speed (or real-time) comparison, collation, and authentication. By using this unique code, it is also possible to realize smaller data size in the transmission of invisible information such as heat source, sound, and X-rays, and information from the five senses. As shown in the conceptual diagram in FIG. 22, by adding a unique code according to each embodiment to what is currently established and recognized as data, such as spatial information, moving images, still images, temperature, and sound, not only can it be linked to the real world, but new ways of using data can be created, and further added value and methods of using data that have already been recorded can be given. For example, in addition to GPS information (location information), direction, angle, temperature, pressure, and the like, various accompanying information such as source and URL already exists in big data. However, when comparing and collating with the real world in real time, the "situation," "state," and "spatial data" necessary for judgment are missing, and the necessary linking cannot be made at present. In contrast, by using the code according to each embodiment to create a link with the reality called "Earth Data," it can play the role of a "key" or trigger for the link between the real world and the system. Since the data size of the code according to each embodiment is very small, a new concept called "byte chain" is provided.

[0107] The present invention is not limited to the contents of the above-mentioned embodiments, and various modifications can be made within the scope of the gist of the present invention. The use and application of the code generated by each embodiment is not limited to the above. The numerical values ​​given in each embodiment are merely examples and are not limited to these. In the first embodiment, the relative difference between each sampling range is calculated, but this may be omitted and the average value of the luminance of each sampling range may be concatenated as character string data.

[0108] In addition, in each of the above-mentioned embodiments, the code is generated by directly concatenating the character string data obtained from the average values ​​or the relative differences, but the form of "concatenation" in the present invention is not limited to this. For example, the character string data obtained by multiplying each average value or each relative difference by a certain coefficient may be concatenated. Furthermore, each character string data may be concatenated in a form in which specific data (delimiter, etc.) is interposed between the character string data corresponding to each average value, etc. Alternatively, character string data corresponding to each average value, etc. may be converted into other character string data under a certain rule and concatenated. In other words, any form of concatenation is included in the concept of "concatenation" in the present invention as long as the relative relationship between the average values, etc. is not destroyed or can be restored. [Explanation of symbols]

[0109] 10, 50: information processing unit, 11, 51: camera, 12, 52: microphone, 13, 53: storage device, 14, 54: operation device, 15, 55: display device, 20, 60: sampling processing unit, 21, 61: feature value calculation unit, 23, 63: encoding processing unit, 24: code recording processing unit, 64: matching processing unit

Claims

1. A method for generating and verifying a code according to the contents of target data using an information processing device, comprising: A first step of obtaining a reference code determined according to reference data; a second step of generating a specific target code corresponding to the specific target data; a third step of comparing said reference code with said specific target code to determine a match / mismatch therebetween; Including, The reference code in the first step is Dividing the reference data into a plurality of first sampling ranges; a step of calculating a first average value of at least one type of first element data among one or more types of first element data included in each of the plurality of first sampling ranges, each of which is expressed by a numerical value, for each of the first sampling ranges; determining a first relative difference for each of the first sampling ranges by dividing the first average value for each of the first sampling ranges by a sum of the first average values ​​for each of the first sampling ranges; generating the reference code corresponding to the reference data by concatenating, as character string data, the first relative difference for each of the first sampling ranges or a predetermined number of digits from the first relative difference; It was previously sought through The identified target code in the second step is Dividing the identified target data into a plurality of second sampling ranges set in the same manner as each of the first sampling ranges; a step of calculating a second average value of at least one type of second element data among one or more types of second element data included in each of the plurality of second sampling ranges, each of which is expressed by a numerical value and which is the same type as the first element data, for each of the second sampling ranges; determining a second relative difference for each of the second sampling ranges by dividing the second average value for each of the second sampling ranges by a sum of the second average values ​​for each of the second sampling ranges; generating the specific target code corresponding to the specific target data by concatenating, as character string data, the second relative difference for each of the second sampling ranges or a predetermined number of digits from the second relative difference; It is generated through The third step is a step of calculating a difference between the first relative difference or a predetermined number of digits from the most significant digit for each of the first sampling ranges obtained based on the reference code and the second relative difference or a predetermined number of digits from the most significant digit for each of the second sampling ranges obtained based on the specific target code, between each of the first sampling ranges and each of the second sampling ranges; determining that the reference code and the specific target code match when each of the differences falls within a predetermined allowable range; A method for matching data, comprising:

2. The number of the first sampling ranges and the number of the second sampling ranges are the same and are 64 or more. The data matching method according to claim 1 .

3. The reference data and the specific target data include still image data, moving image data, or sound data. The data matching method according to claim 1 .

Images