Three-dimensional image search method, device, system and computer readable storage medium

Abstract

The application discloses a three-dimensional image search method, device, system and computer readable storage medium, the method comprises the following steps: obtaining a three-dimensional image to be searched, and generating keyword information based on the three-dimensional image to be searched; obtaining a pre-created storage three-dimensional image information set, and performing parallel optical calculation matching on the storage three-dimensional image information set and the keyword information to obtain a matching result set; determining a search result corresponding to the three-dimensional image to be searched according to the matching result set and the storage three-dimensional image information set; the application generates keyword information based on the three-dimensional image to be searched, performs parallel optical calculation matching on the storage three-dimensional image information set and the keyword information, and determines the search result corresponding to the three-dimensional image to be searched, so that the three-dimensional image with a large amount of data is searched through parallel optical calculation matching, the search time is shortened, and the three-dimensional image search efficiency is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, and particularly relates to a three-dimensional image search method, device, system and computer readable storage medium. BACKGROUND

[0002] With the development of artificial intelligence, VR, AR and metaverse technology, the related application of three-dimensional images will become increasingly common, and the demand for three-dimensional image search technology is increasing.

[0003] The current three-dimensional image search generally adopts a scheme of combining existing computer systems with hardware devices based on electrical signal processing such as conventional CPUs, GPUs or FPGAs. Due to the large amount of data of three-dimensional images, the time required for three-dimensional image search by hardware devices based on electrical signal processing combined with existing computer systems is relatively long, resulting in low efficiency of three-dimensional image search.

[0004] Therefore, how to improve the efficiency of three-dimensional image search is a problem to be solved. SUMMARY

[0005] The main purpose of the present application is to provide a three-dimensional image search method, device, system and computer readable storage medium, which aims to solve the problem of how to improve the efficiency of three-dimensional image search.

[0006] To achieve the above-mentioned purpose, the present application provides a three-dimensional image search method, which comprises the following steps:

[0007] Obtaining a three-dimensional image to be searched, and generating keyword information based on the three-dimensional image to be searched;

[0008] Obtaining a pre-created storage three-dimensional image information set, and performing parallel optical calculation matching on the storage three-dimensional image information set and the keyword information to obtain a matching result set;

[0009] According to the matching result set and the storage three-dimensional image information set, determining the search result corresponding to the three-dimensional image to be searched.

[0010] Optionally, the step of performing parallel optical calculation matching on the storage three-dimensional image information set and the keyword information to obtain a matching result set comprises:

[0011] Selecting a storage three-dimensional image information in the storage three-dimensional image information set in turn, and determining a corresponding light spot matrix of a parallel optical information emission module according to the selected storage three-dimensional image information;

[0012] According to the keyword information, determining a corresponding modulation matrix of a liquid crystal spatial light convolution matrix module;

[0013] perform parallel optical calculation matching based on the light spot matrix and the modulation matrix to determine a matching result of the selected stored three-dimensional image information and the three-dimensional image to be searched;

[0014] until the number of the matching results meets a preset condition, a matching result set is obtained.

[0015] Optionally, the parallel optical information emitting module comprises a multi-byte storage matrix unit and a laser diode matrix unit, and the step of determining the light spot matrix corresponding to the parallel optical information emitting module according to the selected stored three-dimensional image information comprises:

[0016] inputting the selected stored three-dimensional image information into the multi-byte storage matrix unit, generating a driving signal based on the selected stored three-dimensional image information through the multi-byte storage matrix unit, and sending the driving signal to the laser diode matrix unit;

[0017] controlling the laser diodes in the laser diode matrix unit to emit light or extinguish according to the driving signal through the laser diode matrix unit, and obtaining the light spot matrix.

[0018] Optionally, the step of determining the modulation matrix corresponding to the liquid crystal spatial light convolution matrix module according to the keyword information comprises:

[0019] determining the light attenuation amplitude corresponding to each liquid crystal point in the liquid crystal spatial light convolution matrix module according to the keyword information;

[0020] generating the modulation matrix through the liquid crystal spatial light convolution matrix module according to the light attenuation amplitude corresponding to each liquid crystal point.

[0021] Optionally, the step of performing parallel optical calculation matching based on the light spot matrix and the modulation matrix to determine a matching result of the selected stored three-dimensional image information and the three-dimensional image to be searched comprises:

[0022] transmitting a first light signal corresponding to the light spot matrix through the modulation matrix to obtain a second light signal;

[0023] inputting the second light signal into a light signal converging module to generate a third light signal, and sending the third light signal to a photoelectric conversion module through the light signal converging module;

[0024] determining the light signal intensity corresponding to the third light signal through the photoelectric conversion module, and generating a level signal according to the light signal intensity;

[0025] determining a matching result of the selected stored three-dimensional image information and the three-dimensional image to be searched according to the level signal.

[0026] Optionally, the step of determining the search result corresponding to the to-be-searched three-dimensional image according to the matching result set and the stored three-dimensional image information set comprises:

[0027] obtaining a matching degree corresponding to each matching result in the matching result set, and determining a target stored three-dimensional image information set from the stored three-dimensional image information set according to the matching degree and a preset matching condition;

[0028] determining the search result corresponding to the to-be-searched three-dimensional image according to the target stored three-dimensional image information set.

[0029] In addition, to achieve the above object, the application further provides a three-dimensional image search device, which comprises:

[0030] a data processing module, a parallel optical computing matching module and an output module, wherein the data processing module is in communication connection with the parallel optical computing matching module, and the parallel optical computing matching module is in communication connection with the output module;

[0031] the data processing module is configured to acquire a to-be-searched three-dimensional image and generate keyword information based on the to-be-searched three-dimensional image;

[0032] the parallel optical computing matching module is configured to acquire a stored three-dimensional image information set created in advance, and perform parallel optical computing matching on the stored three-dimensional image information set and the keyword information to obtain a matching result set;

[0033] the output module is configured to determine a search result corresponding to the to-be-searched three-dimensional image according to the matching result set and the stored three-dimensional image information set.

[0034] Further, the parallel optical computing matching module comprises a parallel optical information emitting module, a liquid crystal spatial light convolution matrix module, an optical signal converging module, an optoelectronic conversion module and a control module, wherein:

[0035] the data processing module is in communication connection with the parallel optical information emitting module and the control module respectively, the parallel optical information emitting module is in communication connection with the liquid crystal spatial light convolution matrix module, the liquid crystal spatial light convolution matrix module is in communication connection with the optical signal converging module, the optical signal converging module is in communication connection with the optoelectronic conversion module, the optoelectronic conversion module is in communication connection with the control module, and the control module is in communication connection with the liquid crystal spatial light convolution matrix module and the output module respectively;

[0036] The parallel optical information emission module is configured to select stored three-dimensional image information from a stored three-dimensional image information set stored in the data processing module, determine a light spot matrix according to the selected stored three-dimensional image information, and output a first optical signal corresponding to the light spot matrix to the liquid crystal spatial light convolution matrix module;

[0037] The liquid crystal spatial light convolution matrix module is configured to determine a modulation matrix according to keyword information corresponding to the three-dimensional image to be searched sent by the control module, and output a second optical signal obtained based on the modulation matrix and the first optical signal to the optical signal converging module;

[0038] The optical signal converging module is configured to generate a third optical signal according to the second optical signal, and send the third optical signal to the photoelectric conversion module;

[0039] The photoelectric conversion module is configured to determine an optical signal intensity corresponding to the third optical signal, generate a level signal according to the optical signal intensity, and send the level signal to the control module;

[0040] The control module is configured to acquire keyword information generated based on the three-dimensional image to be searched in the data processing module, send the keyword information to the liquid crystal spatial light convolution matrix module, and determine a matching result of each stored three-dimensional image information and the three-dimensional image to be searched according to the level signal, obtain a matching result set, and send the matching result set to the output module.

[0041] Further, the parallel optical information emission module comprises a multi-byte storage matrix unit and a laser diode matrix unit, wherein:

[0042] The multi-byte storage matrix unit is in communication connection with the laser diode matrix unit;

[0043] The multi-byte storage matrix unit is configured to select stored three-dimensional image information from a stored three-dimensional image information set stored in the data processing module, generate a driving signal based on the selected stored three-dimensional image information, and send the driving signal to the laser diode matrix unit;

[0044] The laser diode matrix unit is configured to control a laser diode to emit light or extinguish according to the driving signal, obtain a light spot matrix, and output a first optical signal corresponding to the light spot matrix to the liquid crystal spatial light convolution matrix module.

[0045] Further, the matrix size of the multi-byte storage matrix unit, the laser diode matrix unit and the liquid crystal spatial light convolution matrix module is the same.

[0046] In addition, to achieve the above object, the present application also provides a three-dimensional image search system, comprising a memory, a processor and a three-dimensional image search program stored on the memory and executable on the processor, which realizes the steps of the three-dimensional image search method as described above when executed by the processor.

[0047] In addition, to achieve the above object, the present application also provides a computer readable storage medium, which stores a three-dimensional image search program, which realizes the steps of the three-dimensional image search method as described above when executed by a processor.

[0048] The three-dimensional image search method provided by the present application acquires a three-dimensional image to be searched and generates keyword information based on the three-dimensional image to be searched; acquires a pre-created storage three-dimensional image information set and performs parallel optical calculation matching on the storage three-dimensional image information set and the keyword information to obtain a matching result set; and determines a search result corresponding to the three-dimensional image to be searched according to the matching result set and the storage three-dimensional image information set. The data processing module in the three-dimensional image search device generates keyword information based on the three-dimensional image to be searched, the parallel optical calculation matching module and the output module perform parallel optical calculation matching on the storage three-dimensional image information set and the keyword information to determine the search result corresponding to the three-dimensional image to be searched, and the parallel optical calculation matching module converts the storage three-dimensional image information set and the keyword information into optical information for parallel optical calculation matching, thereby searching the three-dimensional image with a large amount of data. Compared with the existing hardware device based on electrical signal processing combined with the existing computer system for three-dimensional image search, the present application shortens the search time and improves the efficiency of three-dimensional image search. BRIEF DESCRIPTION OF DRAWINGS

[0049] Figure 1 is the device structure schematic diagram of the hardware running environment involved in the embodiment scheme of the present application;

[0050] Figure 2 is the flowchart of the first embodiment of the three-dimensional image search method of the present application;

[0051] Figure 3 is the structure schematic diagram of the three-dimensional image search device of the present application;

[0052] Figure 4 is the structure schematic diagram of the parallel optical calculation matching module in the three-dimensional image search device of the present application;

[0053] Figure 5 is the structure schematic diagram of the parallel optical information emission module in the parallel optical calculation matching module of the present application.

[0054] The implementation, functional features and advantages of the present application will be further described with reference to the embodiments and the accompanying drawings. DETAILED DESCRIPTION

[0055] It should be understood that the specific embodiments described herein merely exemplify the application and do not limit the application.

[0056] As shown in Figure 1 , the device structure of the hardware running environment involved in the embodiment of the present application is shown. Figure 1

[0057] The device of the embodiment of the present application can be a PC or a server device.

[0058] As shown in Figure 1 , the device can include a processor 1001 such as a CPU, a network interface 1004, a user interface 1003, a memory 1005, and a communication bus 1002. The communication bus 1002 is used to realize the connection and communication among the components. The user interface 1003 can include a display screen (Display) and an input unit such as a keyboard (Keyboard). The optional user interface 1003 can further include a standard wired interface and a wireless interface. The network interface 1004 can optionally include a standard wired interface and a wireless interface (such as a WI-FI interface). The memory 1005 can be a high-speed RAM memory or a stable memory (non-volatile memory) such as a disk memory. The memory 1005 can optionally be a storage device independent of the aforementioned processor 1001.

[0059] Those skilled in the art can understand that Figure 1 the device structure shown in the foregoing embodiments does not constitute a limitation on the device, and can include more or fewer components than those shown in the drawings, or combine certain components, or different component arrangements.

[0060] As shown in Figure 1 , the memory 1005 as a computer storage medium can include an operating system, a network communication module, a user interface module, and a three-dimensional image search program.

[0061] The operating system is a program for managing and controlling the portable storage device and software resources, supporting the running of the network communication module, the user interface module, the three-dimensional image search program, and other programs or software; the network communication module is used to manage and control the network interface 1004; and the user interface module is used to manage and control the user interface 1003.

[0062] In Figure 1 ​The storage device shown in the storage device calls a three-dimensional image search program stored in the memory 1005 through the processor 1001, and performs the operations in each embodiment of the three-dimensional image search method described below.

[0063] Based on the hardware structure described above, embodiments of the three-dimensional image search method of the present application are proposed.

[0064] Reference Figure 2 , Figure 2 The flowchart of the first embodiment of the three-dimensional image search method of the present application is shown in the figure. The method comprises:

[0065] Step S10, obtain the three-dimensional image to be searched, and generate keyword information based on the three-dimensional image to be searched;

[0066] Step S20, obtain a pre-created storage three-dimensional image information set, and perform parallel optical calculation matching on the storage three-dimensional image information set and the keyword information to obtain a matching result set;

[0067] Step S30, according to the matching result set and the storage three-dimensional image information set, determine the search result corresponding to the three-dimensional image to be searched.

[0068] The three-dimensional image search method of this embodiment is applied to a three-dimensional image search device of a communication service institution. For the convenience of description, the three-dimensional image search device is taken as an example for illustration. When the three-dimensional image search device obtains the three-dimensional image to be searched input by the user, it generates keyword information based on the three-dimensional image to be searched, and obtains a pre-created storage three-dimensional image information set. The three-dimensional image search device performs parallel optical calculation matching on the storage three-dimensional image information set and the keyword information to obtain a matching result set. According to the matching result set and the storage three-dimensional image information set, the three-dimensional image search device determines the search result corresponding to the three-dimensional image to be searched. It should be noted that the keyword information is the matrix data corresponding to the normalized line segment model corresponding to the three-dimensional image to be searched. Each storage three-dimensional image information in the storage three-dimensional image information set is the matrix data corresponding to the normalized line segment model corresponding to each three-dimensional image in the Internet and the database. Parallel optical calculation matching can be understood as converting the storage three-dimensional image information set and the keyword information into optical information for optical calculation matching, thereby improving the operation efficiency.

[0069] The three-dimensional image search method of the embodiment acquires a three-dimensional image to be searched, and generates keyword information based on the three-dimensional image to be searched; acquires a pre-created three-dimensional image information set, and performs parallel optical calculation matching on the three-dimensional image information set and the keyword information to obtain a matching result set; and determines a search result corresponding to the three-dimensional image to be searched according to the matching result set and the three-dimensional image information set. The three-dimensional image search method of the embodiment generates keyword information based on a three-dimensional image to be searched, performs parallel optical calculation matching on a three-dimensional image information set and keyword information, to determine a search result corresponding to the three-dimensional image to be searched. The three-dimensional image search method of the embodiment searches a three-dimensional image with a large amount of data through parallel optical calculation matching, shortens the time required for searching, and improves the efficiency of three-dimensional image search.

[0070] The following will be described in detail:

[0071] In step S10, a three-dimensional image to be searched is acquired, and keyword information is generated based on the three-dimensional image to be searched.

[0072] In the embodiment, when a three-dimensional image to be searched input by a user is acquired, a three-dimensional image search device generates keyword information based on the three-dimensional image to be searched. The keyword information is three-dimensional matrix data generated according to RGB information of the three-dimensional image to be searched. Specifically, the three-dimensional image search device includes a data processing module, which can be a CPU or a GPU or an FPGA or a DSP. When the three-dimensional image to be searched is input by the user, the three-dimensional image search device normalizes the three-dimensional image to be searched according to n×n×n size information through the data processing module, to obtain a point cloud model of a standard n×n×n size corresponding to the three-dimensional image to be searched, where n is, for example, 255. The three-dimensional image search device identifies each element in the point cloud model as an (R, G, B) three-element vector according to a preset rule of R (0-255), G (0-255), and B (0-255) through the data processing module according to the RGB information of the three-dimensional image to be searched. The three-dimensional image search device adopts a coarse filter to eliminate obvious noise data for the point cloud model of the standard n×n×n size corresponding to the three-dimensional image to be searched through the data processing module, and then adopts a polynomial fitting method to fit the point cloud data into a curve, to obtain a three-dimensional image contour line, to obtain a normalized line segment model corresponding to the three-dimensional image to be searched, and to further discretize the normalized line segment model corresponding to the three-dimensional image to be searched into n×n×n points and store them, to obtain n×n×n matrix data as keyword information and store the keyword information through the data processing module. It should be noted that n is a specific value, which can be set according to actual conditions.

[0073] Step S20, obtaining a pre-created storage three-dimensional image information set, and performing parallel optical calculation matching on the storage three-dimensional image information set and the keyword information to obtain a matching result set.

[0074] In this step, the three-dimensional image search device obtains a pre-created storage three-dimensional image information set, and performs parallel optical calculation matching on the storage three-dimensional image information set and the keyword information to obtain a matching result set. Specifically, before the three-dimensional image search device is put into use, the three-dimensional image search device normalizes a large number of three-dimensional images in the Internet and the database through the data processing module to obtain a standard n x n x n size point cloud model corresponding to each three-dimensional image; the three-dimensional image search device processes the standard n x n x n size point cloud model corresponding to each three-dimensional image through the data processing module to obtain a normalized line segment model corresponding to each three-dimensional image, and further discretizes the normalized line segment model corresponding to each three-dimensional image into n x n x n points and stores them to obtain n x n x n matrix data, and then takes the n x n x n matrix data corresponding to each three-dimensional image as the storage three-dimensional image information, and further obtains the storage three-dimensional image information set and stores the storage three-dimensional image information set through the data processing module. The storage three-dimensional image information set in this embodiment can be understood as information corresponding to a pre-collected three-dimensional image source, which can be used as a sample source for subsequent three-dimensional image search.

[0075] Further, the three-dimensional image search device further includes a parallel optical calculation matching module. After the three-dimensional image search device determines the keyword information corresponding to the to-be-searched three-dimensional image input by the user through the data processing module, the parallel optical calculation matching module selects one storage three-dimensional image information from the storage three-dimensional image information set stored in the data processing module, converts the selected storage three-dimensional image information and the keyword information into corresponding optical signals, and then performs parallel optical calculation matching to obtain the matching result of the selected storage three-dimensional image information and the keyword information; the three-dimensional image search device determines the matching result of each storage three-dimensional image information in the storage three-dimensional image information set and the keyword information through the parallel optical calculation matching module to obtain a matching result set.

[0076] Specifically, the step of performing parallel optical calculation matching on the storage three-dimensional image information set and the keyword information to obtain a matching result set includes:

[0077] Step S201, selecting one storage three-dimensional image information from the storage three-dimensional image information set in sequence, and determining the corresponding light spot matrix of the parallel optical information emitting module according to the selected storage three-dimensional image information;

[0078] In this step, the parallel optical computing matching module in the three-dimensional image searching device comprises a parallel optical information transmitting module. The parallel optical information transmitting module is used to acquire the stored three-dimensional image information set stored in the data processing module, and sequentially select one stored three-dimensional image information from the stored three-dimensional image information set. According to the selected stored three-dimensional image information, the corresponding light spot matrix of the parallel optical information transmitting module is determined.

[0079] Further, the step of determining the light spot matrix corresponding to the parallel optical information transmitting module according to the selected stored three-dimensional image information comprises:

[0080] In step S2011, the selected stored three-dimensional image information is input into the multi-byte storage matrix unit. The multi-byte storage matrix unit generates a driving signal based on the selected stored three-dimensional image information, and sends the driving signal to the laser diode matrix unit.

[0081] In this step, the parallel optical information transmitting module comprises a multi-byte storage matrix unit and a laser diode matrix unit. After the three-dimensional image searching device acquires the stored three-dimensional image information set stored in the data processing module through the parallel optical information transmitting module, one stored three-dimensional image information is sequentially selected from the stored three-dimensional image information set. The selected stored three-dimensional image information is input into the multi-byte storage matrix unit. The multi-byte storage matrix unit generates a driving signal based on the selected stored three-dimensional image information, and sends the driving signal to the laser diode matrix unit. Specifically, the multi-byte storage matrix unit and the laser diode matrix unit are both matrix units with a matrix size of n×n×n, where n is a specific numerical value that can be set according to actual conditions.

[0082] In step S2012, the laser diode matrix unit controls the laser diodes in the laser diode matrix unit to emit light or extinguish according to the driving signal, and obtains a light spot matrix.

[0083] In this step, the three-dimensional image searching device controls the laser diodes in the laser diode matrix unit to emit light or extinguish according to the driving signal sent by the multi-byte storage matrix unit through the laser diode matrix unit, and obtains a light spot matrix. Specifically, the multi-byte storage matrix unit is a matrix unit with a matrix size of n×n×n, so the driving signal sent by the multi-byte storage matrix unit is matrix data with a size of n×n×n. Similarly, the laser diode matrix unit is also a matrix unit with a size of n×n×n, i.e., there are multiple laser diodes distributed in the laser diode matrix unit according to the matrix size of n×n×n. The laser diode matrix unit controls each laser diode to emit light or extinguish according to the received driving signal, and obtains a light spot matrix. It should be noted that the light spot matrix is also matrix data with a size of n×n×n, which is used to represent the optical signal of the stored three-dimensional image information.

[0084] In step S202, a modulation matrix corresponding to the liquid crystal spatial light convolution matrix module is determined according to the keyword information.

[0085] In this step, the parallel optical computing matching module in the three-dimensional image search device includes a control module and a liquid crystal spatial light convolution matrix module, and the control module is in communication connection with the liquid crystal spatial light convolution matrix module. After the three-dimensional image search device determines the keyword information corresponding to the to-be-searched three-dimensional image through the data processing module, the keyword information is input into the control module through the data processing module, a driving signal is generated according to the keyword information through the control module, and the driving signal is sent to the liquid crystal spatial light convolution matrix module. The modulation matrix corresponding to the driving signal is determined through the liquid crystal spatial light convolution matrix module. It should be noted that the driving signal is an n*n*n matrix data, and the modulation matrix is also an n*n*n matrix data. The modulation matrix represents the weighting coefficient vector matrix of the optical signal corresponding to the spot matrix.

[0086] Specifically, step S202 includes:

[0087] In step S2021, the light attenuation amplitude corresponding to each liquid crystal point in the liquid crystal spatial light convolution matrix module is determined according to the keyword information.

[0088] In this step, the liquid crystal spatial light convolution matrix module includes a light-transmitting liquid crystal point array with a matrix size of n*n*n. After the three-dimensional image search device inputs the keyword information into the control module through the data processing module, a driving signal is generated according to the keyword information through the control module, and the driving signal is sent to the liquid crystal spatial light convolution matrix module. The light attenuation amplitude corresponding to each liquid crystal point in the light-transmitting liquid crystal point array in the liquid crystal spatial light convolution matrix module is determined according to the driving signal. It should be noted that the light-transmitting liquid crystal point array is composed of a plurality of liquid crystal points. The light attenuation amplitude of each liquid crystal point is adjusted according to the driving signal by the liquid crystal spatial light convolution matrix module. The light attenuation amplitude can be understood as the light transmittance of the liquid crystal point.

[0089] In step S2022, the modulation matrix is generated through the liquid crystal spatial light convolution matrix module according to the light attenuation amplitude corresponding to each liquid crystal point.

[0090] In this step, the three-dimensional image search device adjusts the light transmittance of each liquid crystal point through the liquid crystal spatial light convolution matrix module according to the light attenuation amplitude corresponding to each liquid crystal point, and then generates the modulation matrix through the light-transmitting liquid crystal point array after the adjustment.

[0091] In step S203, parallel optical computing matching is performed based on the spot matrix and the modulation matrix, and the matching result of the selected stored three-dimensional image information and the to-be-searched three-dimensional image is determined.

[0092] In this step, the three-dimensional image searching device determines the matching result of the selected stored three-dimensional image information and the three-dimensional image to be searched based on the light spot matrix and the modulation matrix through parallel optical calculation matching.

[0093] Specifically, step S203 includes:

[0094] Step S2031, the first light signal corresponding to the light spot matrix is transmitted through the modulation matrix to obtain a second light signal.

[0095] In this step, the three-dimensional image searching device transmits the first light signal corresponding to the light spot matrix generated by the laser diode matrix unit through the modulation matrix corresponding to the liquid crystal spatial light convolution matrix module to obtain a second light signal. Specifically, the laser diode matrix unit controls each laser diode matrix in it to emit light or extinguish to obtain a light spot matrix, and the light spot matrix will generate a corresponding first light signal. The laser diode matrix unit is arranged in parallel with the liquid crystal spatial light convolution matrix module, and at this time, the first light signal will be transmitted through the modulation matrix composed of the light-transmitting liquid crystal dot array in the liquid crystal spatial light convolution matrix module to obtain a second light signal. It should be noted that the first light signal is transmitted through the modulation matrix to obtain a second light signal, which is equivalent to the convolution operation of the first light signal and the modulation matrix to obtain a second light signal. Since the light spot matrix represents the optical signal of the stored three-dimensional image information, and the modulation matrix represents the weighted coefficient vector matrix of the optical signal, the convolution operation of the first light signal and the modulation matrix is equivalent to the convolution operation of the stored three-dimensional image information and the weighted coefficient vector matrix. The present application represents the stored three-dimensional image information and the weighted coefficient vector matrix through optical information, and then performs convolution operation, which improves the efficiency compared with the existing convolution operation through electrical signal.

[0096] Step S2032, the second light signal is input into the light signal converging module to generate a third light signal, and the third light signal is sent to the photoelectric conversion module through the light signal converging module.

[0097] In this step, the 3D image search device includes an optical signal convergence module and a photoelectric conversion module. The 3D image search device inputs the second optical signal into the optical signal convergence module to generate a third optical signal, and then sends the third optical signal to the photoelectric conversion module through the optical signal convergence module. Specifically, the optical signal convergence module is a convex lens. Since both the spot matrix and the modulation matrix are n×n×n matrix data, the generated third optical signal is also an n×n×n matrix data. The second optical signal is input into the optical signal convergence module, which converges the second optical signal to a single point to obtain the third optical signal, which is then sent to the photoelectric conversion module. It should be noted that the first optical signal generated by the spot matrix of the n×n×n laser diode matrix passes through the n×n×n modulation matrix to obtain the second optical signal. The second optical signal is then converged into the third optical signal through the optical signal convergence module, realizing the simulation calculation of optical information. The mathematical meaning of the calculation is that the spot matrix, as the input vector, and the modulation matrix, as the weighting coefficient vector, are multiplied together to achieve a convolution operation. The result of the convolution operation is represented by the total light intensity of the third optical signal.

[0098] Step S2033: Determine the intensity of the optical signal corresponding to the third optical signal through the photoelectric conversion module, and generate a level signal based on the intensity of the optical signal;

[0099] In this step, the three-dimensional image search device sends the third optical signal to the photoelectric conversion module through the optical signal convergence module. The photoelectric conversion module then determines the intensity of the optical signal corresponding to the third optical signal and generates a level signal based on the intensity of the optical signal. Specifically, the photoelectric conversion module may include an optical intensity sensor and a photodiode. The photoelectric conversion module can determine the intensity of the optical signal corresponding to the third optical signal through the optical intensity sensor and convert the third optical signal into a corresponding level signal based on the intensity of the optical signal through the photodiode.

[0100] Step S2034: Based on the level signal, determine the matching result between the selected stored 3D image information and the 3D image to be searched.

[0101] In this step, the 3D image search device includes a control module, which can acquire relevant information generated by each module in the above process. The 3D image search device, through the control module, acquires the spot matrix, modulation matrix, photoelectric conversion function of the photoelectric conversion module, and light signal intensity difference coefficient. Based on the spot matrix, modulation matrix, photoelectric conversion function of the photoelectric conversion module, and light signal intensity difference coefficient, it calculates the amplitude of the level signal and determines the matching result between the selected stored 3D image information and the 3D image to be searched based on the amplitude of the level signal.

[0102] Specifically, the amplitude V of the level signal o =L(n)M(h(u1)*f(u2), where, Vo is the amplitude of the level signal; L(n) is the photoelectric conversion function of the photoelectric conversion module, which represents the conversion capability of the light intensity to the power amplitude; M is the light signal intensity difference coefficient, in the actual system, there is a linear difference between the total light intensity of the point light and the total light intensity under the ideal light condition, and the difference can be represented by the difference coefficient M; h(u1) is the modulation matrix; f(u2) is the light spot matrix; and * is a convolution operator.

[0103] Further, the amplitude V of the level signal o and the matching level V p The difference P o = (V o -V p ), P o is the matching result, and the smaller P o is, the higher the matching degree of the selected storage three-dimensional image information and the three-dimensional image to be searched is. Wherein, each storage three-dimensional image information in the pre-created storage three-dimensional image information set corresponds to a matching level, and the matching level is determined in the same way as the determination of the level signal corresponding to the above-mentioned three-dimensional image to be searched, and the matching level is determined in advance and stored.

[0104] Step S204, until the number of matching results meets the preset condition, a matching result set is obtained.

[0105] In this step, the three-dimensional image search device performs the above parallel optical calculation matching step on the mass storage three-dimensional image information in the storage three-dimensional image information set and the keyword information, until the number of matching results meets the preset condition. Alternatively, the three-dimensional image search device counts the matching degree corresponding to each matching result that has been determined, and counts the number of matching results whose matching degree is greater than a preset matching degree threshold. When the number of matching results whose matching degree is greater than the preset threshold is greater than a preset number threshold, it is determined that the number of matching results meets the preset condition, and a matching result set is obtained. Alternatively, the three-dimensional image search device performs the above parallel optical calculation matching step on each storage three-dimensional image information in the storage three-dimensional image information set and the keyword information, until the matching result of each storage three-dimensional image information in the storage three-dimensional image information set and the three-dimensional image to be searched is determined, and a matching result set is obtained.

[0106] Step S30, according to the matching result set and the storage three-dimensional image information set, determining the search result corresponding to the three-dimensional image to be searched.

[0107] In this embodiment, the three-dimensional image search device includes an output module; the three-dimensional image search device determines the search result corresponding to the three-dimensional image to be searched according to the matching result set and the storage three-dimensional image information set through the output module.

[0108] Specifically, step S30 comprises:

[0109] Step S301, obtaining a matching degree corresponding to each matching result in the matching result set, and determining a target storage three-dimensional image information set from the storage three-dimensional image information set according to the matching degree and a preset matching condition;

[0110] In this step, the three-dimensional image search device obtains a matching degree corresponding to each matching result in the matching result set through the output module, and determines a target storage three-dimensional image information set from the storage three-dimensional image information set according to the matching degree and a preset matching condition; specifically, considering the inexact matching in the search, an inexact matching coefficient is set to represent the difference between the three-dimensional image to be searched and the selected storage three-dimensional image information, and the matching degree corresponding to each matching result is compared with the inexact matching coefficient, and the selected storage three-dimensional image information corresponding to the matching result with a matching degree not less than the inexact matching coefficient is selected from the storage three-dimensional image information set as the target storage three-dimensional image information set.

[0111] Step S302, determining a search result corresponding to the three-dimensional image to be searched according to the target storage three-dimensional image information set.

[0112] In this step, the three-dimensional image search device determines a search result corresponding to the three-dimensional image to be searched according to the target storage three-dimensional image information set through the output module. Optionally, a storage three-dimensional image set corresponding to the target storage three-dimensional image information set is directly determined as the search result corresponding to the three-dimensional image to be searched; optionally, each storage three-dimensional image information in the target storage three-dimensional image information set is sorted in descending order according to the matching degree of the corresponding matching result, and a storage three-dimensional image set corresponding to a plurality of storage three-dimensional image information in the front according to the sorting result is selected to determine as the search result corresponding to the three-dimensional image to be searched.

[0113] The three-dimensional image search device of the embodiment acquires a three-dimensional image to be searched, generates keyword information based on the three-dimensional image to be searched, acquires a pre-created storage three-dimensional image information set, and performs parallel optical calculation matching on the storage three-dimensional image information set and the keyword information to obtain a matching result set; and determines a search result corresponding to the three-dimensional image to be searched according to the matching result set and the storage three-dimensional image information set. The data processing module in the three-dimensional image search device generates keyword information based on the three-dimensional image to be searched, the parallel optical calculation matching module and the output module perform parallel optical calculation matching on the storage three-dimensional image information set and the keyword information to determine the search result corresponding to the three-dimensional image to be searched. The parallel optical calculation matching module converts the storage three-dimensional image information set and the keyword information into optical information for parallel optical calculation matching. The three-dimensional image search method using parallel simulation light calculation completes the calculation of the convolution operation, Fourier transform and other calculation processes through optical structures at one time, and then searches the three-dimensional image with a large amount of data. Compared with the existing hardware device based on electrical signal processing and the existing computer system for three-dimensional image search, the search time is shortened and the efficiency of three-dimensional image search is improved.

[0114] Further, with reference to Figure 3 The application further provides a three-dimensional image search device, which comprises:

[0115] a data processing module, a parallel optical calculation matching module and an output module, wherein the data processing module is in communication connection with the parallel optical calculation matching module, and the parallel optical calculation matching module is in communication connection with the output module.

[0116] The data processing module can be a CPU or a GPU or an FPGA or a DSP, used to obtain a three-dimensional image to be searched and generate keyword information based on the three-dimensional image to be searched. Specifically, when a three-dimensional image to be searched is input by a user, the three-dimensional image search device normalizes the three-dimensional image to be searched according to the size information of n x n x n through the data processing module, to obtain a point cloud model of a standard n x n x n size corresponding to the three-dimensional image to be searched. The three-dimensional image search device identifies each element in the point cloud model as an (R, G, B) three-element vector according to the RGB information of the three-dimensional image to be searched and a preset rule of R (0-255), G (0-255), and B (0-255) through the data processing module. The three-dimensional image search device adopts a coarse filtering method to remove obvious noise data from the point cloud model of the standard n x n x n size corresponding to the three-dimensional image to be searched through the data processing module, and then adopts a polynomial fitting method to fit the point cloud data into a curve, thereby obtaining a three-dimensional image contour line and a normalized line segment model corresponding to the three-dimensional image to be searched. The normalized line segment model corresponding to the three-dimensional image to be searched is further discretized into n x n x n points and stored, to obtain n x n x n matrix data as keyword information and store the keyword information through the data processing module. It should be noted that n is a specific value, which can be set according to actual conditions.

[0117] Further, before the three-dimensional image search device is put into use, the data processing module normalizes each three-dimensional image in the Internet and a database to obtain a point cloud model of a standard n x n x n size corresponding to each three-dimensional image. The three-dimensional image search device processes the point cloud model of the standard n x n x n size corresponding to each three-dimensional image through the data processing module to obtain a normalized line segment model corresponding to each three-dimensional image. The normalized line segment model corresponding to each three-dimensional image is further discretized into n x n x n points and stored to obtain n x n x n matrix data, which is then stored as three-dimensional image information corresponding to each three-dimensional image. Thus, a set of stored three-dimensional image information is obtained and stored through the data processing module.

[0118] The parallel optical computing matching module is used to sequentially select one piece of stored three-dimensional image information from the set of stored three-dimensional image information stored in the data processing module, convert the selected stored three-dimensional image information and the keyword information into corresponding optical signals, and then perform parallel optical computing matching to obtain a matching result of the selected stored three-dimensional image information and the keyword information. The three-dimensional image search device determines the matching result of each piece of stored three-dimensional image information and the keyword information in the set of stored three-dimensional image information through the parallel optical computing matching module, to obtain a set of matching results.

[0119] An output module is configured to determine a search result corresponding to the to-be-searched three-dimensional image according to the matching result set and the stored three-dimensional image information set.

[0120] Further, with reference to Figure 4 The parallel optical computation matching module in the three-dimensional image search device comprises a parallel optical information emitting module, a liquid crystal spatial light convolution matrix module, a light signal converging module, a photoelectric conversion module and a control module.

[0121] The data processing module is in communication connection with the parallel optical information emitting module and the control module, the parallel optical information emitting module is in communication connection with the liquid crystal spatial light convolution matrix module, the liquid crystal spatial light convolution matrix module is in communication connection with the light signal converging module, the light signal converging module is in communication connection with the photoelectric conversion module, the photoelectric conversion module is in communication connection with the control module, and the control module is in communication connection with the liquid crystal spatial light convolution matrix module and the output module.

[0122] The parallel optical information emitting module is configured to select stored three-dimensional image information from the stored three-dimensional image information set stored in the data processing module, determine a light spot matrix according to the selected stored three-dimensional image information, and output a first light signal corresponding to the light spot matrix to the liquid crystal spatial light convolution matrix module.

[0123] The liquid crystal spatial light convolution matrix module comprises a light-transmitting liquid crystal dot array with a matrix size of n x n x n, and is configured to determine, according to a driving signal generated by the control module according to keyword information corresponding to the to-be-searched three-dimensional image, an optical attenuation amplitude corresponding to each liquid crystal dot in the light-transmitting liquid crystal dot array.

[0124] Further, the liquid crystal spatial light convolution matrix module adjusts the light-transmitting rate of each liquid crystal dot according to the optical attenuation amplitude corresponding to each liquid crystal dot, and generates a modulation matrix through the adjusted light-transmitting liquid crystal dot array.

[0125] Further, the liquid crystal spatial light convolution matrix module receives the first light signal sent by the parallel optical information emitting module, transmits the first light signal through the modulation matrix to obtain a second light signal, and outputs the second light signal to the light signal converging module.

[0126] The light signal converging module can be a convex lens or other light-converging element, configured to converge the second light signal to a point to obtain a third light signal, and send the third light signal to the photoelectric conversion module.

[0127] The photoelectric conversion module can include a light intensity sensor and a photodiode, the photoelectric conversion module can determine the light signal intensity corresponding to the third light signal through the light intensity sensor, and convert the third light signal into a corresponding level signal according to the light signal intensity through the photodiode, and send the level signal to the control module.

[0128] The control module is configured to acquire keyword information generated based on the to-be-searched three-dimensional image in the data processing module, send the keyword information to the liquid crystal spatial light convolution matrix module, and determine a matching result of each stored three-dimensional image information with the to-be-searched three-dimensional image according to the level signal sent by the photoelectric conversion module, obtain a matching result set, and send the matching result set to the output module.

[0129] Further, referring to Figure 5 The parallel optical information emission module in the parallel optical computing matching module in the three-dimensional image search device includes a multi-byte storage matrix unit and a laser diode matrix unit. The matrix size of the multi-byte storage matrix unit, the laser diode matrix unit and the liquid crystal spatial light convolution matrix module is the same, and the matrix size is n×n×n, where n is a specific value, which can be set according to actual conditions.

[0130] The multi-byte storage matrix unit and the laser diode matrix unit are in communication connection.

[0131] The multi-byte storage matrix unit is configured to select stored three-dimensional image information from the stored three-dimensional image information set stored in the data processing module and cache the selected stored three-dimensional image information, generate a driving signal based on the selected stored three-dimensional image information, and send the driving signal to the laser diode matrix unit.

[0132] The laser diode matrix unit includes a plurality of laser diodes distributed according to the matrix size of n×n×n. The laser diode matrix unit is configured to control each laser diode in the laser diode matrix unit to emit light or be extinguished according to the driving signal sent by the multi-byte storage matrix unit, obtain a light spot matrix, and output a first light signal corresponding to the light spot matrix to the liquid crystal spatial light convolution matrix module.

[0133] The application further provides a three-dimensional image search system.

[0134] The three-dimensional image search system includes a memory, a processor, and a three-dimensional image search program stored on the memory and executable on the processor. When the three-dimensional image search program is executed by the processor, the steps of the three-dimensional image search method are implemented.

[0135] The method implemented when the three-dimensional image search program running on the processor is executed can refer to the embodiments of the three-dimensional image search method of the application, which will not be described here.

[0136] The present application also provides a computer readable storage medium.

[0137] The computer readable storage medium stores a three-dimensional image search program, and the three-dimensional image search program, when executed by a processor, implements the steps of the three-dimensional image search method.

[0138] The method implemented when the three-dimensional image search program running on the processor is executed can refer to the embodiments of the three-dimensional image search method of the present application, which will not be described here.

[0139] It should be noted that, in this document, the terms "comprises", "comprising", or any other variant thereof are intended to cover a non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements does not include only those elements recited, but can also include other elements not expressly listed or inherent to such process, method, article, or system. Without more limitations, an element defined by the statement "comprising a" does not exclude the existence of additional identical elements in the process, method, article, or system that includes the element.

[0140] The above-mentioned embodiment numbers of the present application are only for description, and do not represent the advantages or disadvantages of the embodiments.

[0141] From the above description of the embodiments, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be realized by means of software and necessary general hardware platforms, of course, they can also be realized by hardware, but in many cases the former is a better embodiment. Based on such understanding, the technical solutions of the present application or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as a ROM / RAM, a magnetic disk, an optical disk) and includes a number of instructions for making a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) execute the methods described in the various embodiments of the present application.

[0142] The above is only the preferred embodiment of the present application, and does not limit the patent scope of the present application, and any equivalent structure or equivalent process transformation, or direct or indirect application in other related technical fields, is also included in the patent protection scope of the present application.

Claims

1. A three-dimensional image search method characterized by, The three-dimensional image searching method comprises the following steps: acquiring a three-dimensional image to be searched, and generating keyword information based on the three-dimensional image to be searched; acquiring a pre-created three-dimensional image information set, and performing parallel optical calculation matching on the three-dimensional image information set and the keyword information to obtain a matching result set; determining a search result corresponding to the three-dimensional image to be searched according to the matching result set and the three-dimensional image information set; the step of performing parallel optical calculation matching on the three-dimensional image information set and the keyword information to obtain a matching result set comprises: selecting a three-dimensional image information in the three-dimensional image information set in sequence, and determining a light spot matrix corresponding to a parallel optical information emitting module according to the selected three-dimensional image information; determining a modulation matrix corresponding to a liquid crystal spatial light convolution matrix module according to the keyword information; performing parallel optical calculation matching based on the light spot matrix and the modulation matrix to determine a matching result of the selected three-dimensional image information and the three-dimensional image to be searched; until the number of matching results meets a preset condition, a matching result set is obtained.

2. The three-dimensional image search method according to claim 1, wherein the parallel optical information emitting module comprises a multi-byte storage matrix unit and a laser diode matrix unit, and the step of determining a light spot matrix corresponding to a parallel optical information emitting module according to the selected three-dimensional image information comprises: inputting the selected three-dimensional image information into the multi-byte storage matrix unit, generating a driving signal based on the selected three-dimensional image information through the multi-byte storage matrix unit, and sending the driving signal to the laser diode matrix unit; controlling the laser diodes in the laser diode matrix unit to emit light or extinguish according to the driving signal through the laser diode matrix unit to obtain a light spot matrix.

3. The three-dimensional image search method according to Claim 1, wherein the step of determining a modulation matrix corresponding to a liquid crystal spatial light convolution matrix module according to the keyword information comprises: determining a light attenuation amplitude corresponding to each liquid crystal point in the liquid crystal spatial light convolution matrix module according to the keyword information; generating a modulation matrix through the liquid crystal spatial light convolution matrix module according to the light attenuation amplitude corresponding to each liquid crystal point.

4. The three-dimensional image search method according to claim 1, wherein the step of performing parallel optical calculation matching based on the light spot matrix and the modulation matrix to determine a matching result of the selected three-dimensional image information and the three-dimensional image to be searched comprises: transmitting a first light signal corresponding to the light spot matrix through the modulation matrix to obtain a second light signal; inputting the second light signal into a light signal converging module to generate a third light signal, and sending the third light signal to a photoelectric conversion module through the light signal converging module; determining a light signal intensity corresponding to the third light signal through the photoelectric conversion module, and generating a level signal according to the light signal intensity; determining a matching result of the selected three-dimensional image information and the three-dimensional image to be searched according to the level signal.

5. The three-dimensional image search method according to Claim 1, wherein the step of determining a search result corresponding to the three-dimensional image to be searched according to the matching result set and the three-dimensional image information set comprises: Acquire the matching degree corresponding to each matching result in the matching result set, and determine a target storage three-dimensional image information set from the storage three-dimensional image information set according to the matching degree and a preset matching condition; Determine a search result corresponding to the three-dimensional image to be searched according to the target storage three-dimensional image information set.

6. A three-dimensional image search apparatus characterized by comprising: The three-dimensional image searching device comprises: A data processing module, a parallel optical computing and matching module, and an output module, wherein the data processing module is in communication connection with the parallel optical computing and matching module, and the parallel optical computing and matching module is in communication connection with the output module; The data processing module is configured to acquire a three-dimensional image to be searched and generate keyword information based on the three-dimensional image to be searched; The parallel optical computing and matching module is configured to acquire a storage three-dimensional image information set created in advance, and perform parallel optical computing and matching on the storage three-dimensional image information set and the keyword information to obtain a matching result set; The output module is configured to determine a search result corresponding to the three-dimensional image to be searched according to the matching result set and the storage three-dimensional image information set; The parallel optical computing and matching on the storage three-dimensional image information set and the keyword information to obtain the matching result set comprises: Selecting a storage three-dimensional image information from the storage three-dimensional image information set in sequence, and determining a light spot matrix corresponding to a parallel optical information emitting module according to the selected storage three-dimensional image information; Determining a modulation matrix corresponding to a liquid crystal spatial light convolution matrix module according to the keyword information; Performing parallel optical computing and matching based on the light spot matrix and the modulation matrix to determine a matching result of the selected storage three-dimensional image information and the three-dimensional image to be searched; Until the number of matching results meets a preset condition, the matching result set is obtained.

7. The three-dimensional image search apparatus according to claim 6, wherein The parallel optical computing and matching module comprises a parallel optical information emitting module, a liquid crystal spatial light convolution matrix module, an optical signal converging module, an optoelectronic conversion module, and a control module, wherein: The data processing module is in communication connection with the parallel optical information emitting module and the control module respectively, the parallel optical information emitting module is in communication connection with the liquid crystal spatial light convolution matrix module, the liquid crystal spatial light convolution matrix module is in communication connection with the optical signal converging module, the optical signal converging module is in communication connection with the optoelectronic conversion module, the optoelectronic conversion module is in communication connection with the control module, and the control module is in communication connection with the liquid crystal spatial light convolution matrix module and the output module respectively; The parallel optical information emitting module is configured to select a storage three-dimensional image information from a storage three-dimensional image information set stored in the data processing module, determine a light spot matrix according to the selected storage three-dimensional image information, and output a first optical signal corresponding to the light spot matrix to the liquid crystal spatial light convolution matrix module; The liquid crystal spatial light convolution matrix module is configured to determine a modulation matrix according to the keyword information corresponding to the three-dimensional image to be searched sent by the control module, and output a second light signal based on the modulation matrix and the first light signal to the light signal converging module; The light signal converging module is configured to generate a third light signal according to the second light signal, and send the third light signal to the photoelectric conversion module; The photoelectric conversion module is configured to determine a light signal intensity corresponding to the third light signal, and generate a level signal according to the light signal intensity, and send the level signal to the control module; The control module is configured to acquire keyword information generated based on the three-dimensional image to be searched in the data processing module, send the keyword information to the liquid crystal spatial light convolution matrix module, and determine a matching result of each stored three-dimensional image information and the three-dimensional image to be searched according to the level signal, obtain a matching result set, and send the matching result set to the output module.

8. The three-dimensional image search apparatus according to claim 7, wherein The parallel optical information emission module comprises a multi-byte storage matrix unit and a laser diode matrix unit, wherein: The multi-byte storage matrix unit is in communication connection with the laser diode matrix unit; The multi-byte storage matrix unit is configured to select stored three-dimensional image information from a stored three-dimensional image information set stored in the data processing module, generate a driving signal based on the selected stored three-dimensional image information, and send the driving signal to the laser diode matrix unit; The laser diode matrix unit is configured to control the laser diode to emit light or extinguish according to the driving signal, obtain a light spot matrix, and output a first light signal corresponding to the light spot matrix to the liquid crystal spatial light convolution matrix module.

9. The three-dimensional image search apparatus according to claim 8, wherein The matrix size of the multi-byte storage matrix unit, the laser diode matrix unit and the liquid crystal spatial light convolution matrix module is the same.

10. A three-dimensional image search system characterized by comprising: The three-dimensional image search system comprises a memory, a processor and a three-dimensional image search program stored on the memory and executable on the processor, and the three-dimensional image search program implements the steps of the three-dimensional image search method according to any one of claims 1 to 5 when executed by the processor.

11. A computer readable storage medium, characterized in that, The computer readable storage medium stores a three-dimensional image search program, and the three-dimensional image search program implements the steps of the three-dimensional image search method according to any one of claims 1 to 5 when executed by the processor.

Images