Image search test method, device, product and storage medium based on multiple scenarios

Abstract

The present application relates to the field of deep learning and artificial intelligence technology, and discloses a method for image search based on multiple scenarios, including selecting data of the same category from a database to obtain an initial data group; selecting a retrieval data group from the initial data group; loading the data of the initial data group and the retrieval data group into the memory through the LOAD_DATA method in the test script; extracting features from the data of the retrieval data group through an improved RepVGG algorithm; comparing the features to obtain test results; and outputting the test results to generate a test report. This method improves the efficiency of retesting, makes the original algorithm model more practical in specific scenarios, and improves the accuracy of feature extraction. The present application also discloses a device, product, and storage medium for image search based on multiple scenarios.

Description

Technical Field

[0001] The present application relates to the field of deep learning and artificial intelligence technology, and for example, to a method, device, product and storage medium for image search based on multiple scenarios. Background Art

[0002] At present, the image data information on the Internet is becoming increasingly large, and users' requirements for image search are also constantly increasing. Person re-identification, also known as pedestrian re-identification, is a technology that uses computer vision technology to determine whether there is a specific pedestrian in an image or video sequence.

[0003] In existing image search testing schemes, the ReID algorithm is usually used.

[0004] In the process of implementing the embodiments of the present disclosure, it is found that there are at least the following problems in the related art:

[0005] Since actual scenarios are numerous and complex, existing ReID algorithms cannot adapt well to specific scenarios. Summary of the invention

[0006] In order to provide a basic understanding of some aspects of the disclosed embodiments, a brief summary is given below. The summary is not an extensive review, nor is it intended to identify key / critical components or delineate the scope of protection of these embodiments, but rather serves as a prelude to the detailed description that follows.

[0007] The embodiments of the present disclosure provide a method, device, product and storage medium for image search based on multiple scenarios to solve the technical problems of multi-scenario applications.

[0008] In some embodiments, the method comprises:

[0009] Select data of the same category from the database to obtain an initial data group, wherein the number of the initial data group ranges from [500,000 to 1.5 million] groups;

[0010] Selecting a search data group from the initial data group, the number of the search data groups is in the range of [20, 60] groups, wherein each search data group includes a number of pictures in the range of [10, 30], and the pictures include different angles, cameras and points;

[0011] Load the data of the initial data group and the search data group into the memory through the LOAD_DATA method in the test script;

[0012] After merging some convolutional layers and BN layers by improving the RepVGG algorithm and adding depthwise separable convolution to replace some convolutional layers in the RepVGG algorithm, features in the data of the initial data group and the retrieval data group are extracted;

[0013] Comparing the features to obtain test results;

[0014] Output the test results to generate a test report.

[0015] Specifically, the depth-wise separable convolution includes depth-wise convolution and point-wise convolution, and the computational complexity of the depth-wise separable convolution is:

[0016] D k 2 *D p 2 *M+N*D p 2 *M (1), where D k is the size of the convolution kernel, D p Indicates that each convolution kernel slides vertically and horizontally D p times, N is the number of convolution kernels, and M is the number of channels of the convolution kernel.

[0017] Specifically, the method extracts features from the data of the initial data group and the search data group by improving the RepVGG algorithm to merge some convolutional layers and BN layers and adding depthwise separable convolution to replace some convolutional layers in the RepVGG algorithm, and further includes:

[0018] CircleLoss is added to the RepVGG algorithm network for training. The CircleLoss formula is:

[0019] Among them, s p is the intra-class similarity, s n is the out-of-class similarity, i, j, n and p are natural numbers greater than or equal to 1, L and K are the number of in-class and out-of-class targets respectively, γ is the expansion factor, and m is the relaxation factor.

[0020] Specifically, the comparing the features to obtain the test results includes:

[0021] The feature comparison algorithm is used to compare the features to obtain the test results. The feature comparison algorithm formula is:

[0022]

[0023] Among them, A and B are vector eigenvalues, λ and ρ are weight thresholds, i, j, n and ρ are natural numbers greater than or equal to 1, and xi and i are the coordinates of the feature points.

[0024] Specifically, the outputting the test results to generate a test report includes:

[0025] Unifying the format of the test results to obtain test results in a unified format;

[0026] The historical records of the test results in the unified format are backed up and a test report is generated.

[0027] In some embodiments, the apparatus comprises:

[0028] An initial data group selection module is configured to select data of the same category from a database to obtain an initial data group, wherein the number of the initial data groups ranges from [500,000 to 1.5 million] groups;

[0029] A retrieval data group selection module is configured to select a retrieval data group from the initial data group, wherein the number of the retrieval data groups ranges from [20, 60] groups, wherein each retrieval data group includes pictures in the number range of [10, 30], and the pictures include different angles, cameras and points;

[0030] A loading module, configured to load the data of the initial data group and the retrieved data group into a memory through a LOAD_DATA method in a test script;

[0031] A feature extraction module is configured to extract features from the data of the initial data group and the retrieval data group by improving the RepVGG algorithm to merge some convolutional layers and BN layers and adding depthwise separable convolution to replace some convolutional layers in the RepVGG algorithm;

[0032] A feature comparison module is configured to compare the features to obtain a test result;

[0033] The generating module is configured to output the test result and generate a test report.

[0034] In some embodiments, the apparatus comprises:

[0035] A processor and a memory storing program instructions, characterized in that the processor is configured to execute the above-mentioned multi-scenario based image search testing method when running the program instructions.

[0036] In some embodiments, the product comprises:

[0037] The image search testing device based on multiple scenarios as described above.

[0038] In some embodiments, the storage medium stores program instructions, and when the program instructions are run, the image search testing method based on multiple scenarios as described above is executed.

[0039] The image search test method, device, product and storage medium based on multiple scenarios provided by the embodiments of the present disclosure store the initial data group and the retrieval data group in the memory, so that the intermediate process files can be saved locally, thereby improving the efficiency of re-testing. At the same time, the improved RepVGG algorithm adds a depthwise separable convolution to replace some convolution layers in the RepVGG algorithm, thereby making the original network model more practical in specific scenarios and improving the feature extraction accuracy of the traditional RepVGG algorithm.

[0040] The above general description and the following description are exemplary and explanatory only and are not intended to limit the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] One or more embodiments are exemplarily described by corresponding drawings, which do not limit the embodiments. Elements with the same reference numerals in the drawings are shown as similar elements, and the drawings do not constitute a scale limitation, and wherein:

[0042] Figure 1 is a schematic diagram of a multi-scenario based image search testing method provided by an embodiment of the present disclosure;

[0043] Figure 2 is a schematic diagram of a multi-scenario based image search testing device provided by an embodiment of the present disclosure;

[0044] Figure 3 It is a schematic diagram of another image search testing device based on multiple scenarios provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0045] In order to be able to understand the features and technical contents of the embodiments of the present disclosure in more detail, the implementation of the embodiments of the present disclosure is described in detail below in conjunction with the accompanying drawings. The attached drawings are for reference only and are not used to limit the embodiments of the present disclosure. In the following technical description, for the convenience of explanation, a full understanding of the disclosed embodiments is provided through multiple details. However, one or more embodiments can still be implemented without these details. In other cases, to simplify the drawings, well-known structures and devices can be simplified for display.

[0046] The terms "first", "second", etc. in the specification and claims of the disclosed embodiments and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the terms used in this way can be interchanged where appropriate, so that the embodiments of the disclosed embodiments described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions.

[0047] Unless otherwise stated, the term "plurality" means two or more.

[0048] In the embodiment of the present disclosure, the character " / " indicates that the preceding and following objects are in an "or" relationship. For example, A / B indicates: A or B.

[0049] The term "and / or" is a description of the association relationship between objects, indicating that three relationships can exist. For example, A and / or B means: A or B, or, A and B.

[0050] The term "correspondence" may refer to an association relationship or a binding relationship. The correspondence between A and B means that there is an association relationship or a binding relationship between A and B.

[0051] Combination Figure 1 As shown, the embodiment of the present disclosure provides a method for searching images by image based on multiple scenarios, including:

[0052] S01, selecting data of the same category from a database to obtain an initial data group, wherein the number of the initial data group ranges from [500,150] ten thousand groups, and the data of the same category includes one of pedestrian data, two-wheeled vehicle data, three-wheeled vehicle data or four-wheeled vehicle data;

[0053] S02, selecting a search data group from the initial data group, the number of the search data groups being in the range of [20, 60] groups, wherein each search data group includes pictures in the range of [10, 30], and the pictures include different angles, cameras and points;

[0054] S03, loading the data of the initial data group and the searched data group into the memory through the LOAD_DATA method in the test script;

[0055] S04, extracting features from the data of the initial data group and the retrieval data group by improving the RepVGG algorithm to merge some convolutional layers and BN layers and adding depthwise separable convolution to replace some convolutional layers in the RepVGG algorithm;

[0056] S05, comparing the features to obtain test results;

[0057] S06, output the test results to generate a test report.

[0058] Optionally, in step S04, the depthwise separable convolution includes depthwise convolution and pointwise convolution, and the computational complexity of the depthwise separable convolution is:

[0059] D k 2 *D p 2 *M+N*D p 2 *M (1), where D k is the size of the convolution kernel, D p Indicates that each convolution kernel slides vertically and horizontally D p times, N is the number of convolution kernels, and M is the number of channels of the convolution kernel. The amount of calculation for ordinary convolution is: D k 2 *D p 2 *M*N, It can be concluded that depth-wise separable convolution greatly reduces the number of parameters and improves the speed of calculation.

[0060] Optionally, step S04 further includes adding CircleLoss to the RepVGG algorithm network for training. The CircleLoss formula is:

[0061] Among them, s p is the intra-class similarity, s n is the out-of-class similarity, i, j, n and p are natural numbers greater than or equal to 1, L and K are the number of in-class and out-of-class targets, γ is the expansion factor, and m is the relaxation factor. The addition of CircleLoss enables the RepVGG algorithm network to better distinguish the differences between different categories and the correlations between the same categories.

[0062] Optionally, step S05 includes:

[0063] The feature comparison algorithm is used to compare the features to obtain the test results. The feature comparison algorithm formula is:

[0064]

[0065] Among them, A and B are vector eigenvalues, λ and ρ are weight thresholds, which are generally set to 0.6 and 0.4, i, j, n and ρ are natural numbers greater than or equal to 1, and x i and iis the coordinate of the feature point. Compared with the traditional feature comparison algorithm, the weight threshold is added to make the result of feature comparison more accurate. The smaller the value of Similarity, the more similar the two targets are, and the larger the value, the greater the probability that they are not the same target.

[0066] Optionally, step S06 includes:

[0067] Unifying the format of the test results to obtain test results in a unified format;

[0068] The historical records of the test results in the above unified format are backed up and a test report is generated.

[0069] By unifying the test results into a unified format and storing the historical records to generate a test report, the test results can be compared with previous results, making the evaluation of the test results more convenient.

[0070] Combination Figure 2 As shown, the embodiment of the present disclosure provides a multi-scenario based image search test device, including an initial data set selection module 21, a retrieval data set selection module 22, a loading module 23, a feature extraction module, a feature comparison module and a generation module.

[0071] The initial data group selection module 21 is configured to select data of the same category from the database to obtain an initial data group, wherein the number of the initial data groups ranges from [500,000 to 1.5 million] groups;

[0072] The retrieval data group selection module 22 is configured to select a retrieval data group from the initial data group, wherein the number of the retrieval data groups ranges from [20, 60] groups, wherein each retrieval data group includes pictures in the number range of [10, 30], and the pictures include different angles, cameras and points;

[0073] A loading module 23, configured to load the data of the initial data group and the searched data group into the memory through the LOAD_DATA method in the test script;

[0074] A feature extraction module 24 is configured to extract features from the data of the initial data set and the search data set by improving the RepVGG algorithm to merge some convolutional layers and BN layers and adding depthwise separable convolution to replace some convolutional layers in the RepVGG algorithm;

[0075] A feature comparison module 25 is configured to compare the features to obtain a test result;

[0076] The generating module 26 is configured to output the test result and generate a test report.

[0077] The image search testing device based on multiple scenarios provided by the embodiment of the present disclosure improves the efficiency of re-testing, while making the original network model more practical in specific scenarios and improving the feature extraction accuracy of the traditional RepVGG algorithm.

[0078] Combination Figure 3 As shown, the embodiment of the present disclosure provides a multi-scenario based image search test device, including a processor (processor) 100 and a memory (memory) 101. Optionally, the device may also include a communication interface (Communication Interface) 102 and a bus 103. Among them, the processor 100, the communication interface 102, and the memory 101 can communicate with each other through the bus 103. The communication interface 102 can be used for information transmission. The processor 100 can call the logic instructions in the memory 101 to execute the method for ... of the above embodiment.

[0079] In addition, the logic instructions in the memory 101 described above may be implemented in the form of software functional units and when sold or used as independent products, may be stored in a computer-readable storage medium.

[0080] The memory 101, as a computer-readable storage medium, can be used to store software programs and computer executable programs, such as program instructions / modules corresponding to the method in the embodiment of the present disclosure. The processor 100 executes functional applications and data processing by running the program instructions / modules stored in the memory 101, that is, implements the method for image search testing based on multiple scenarios in the above-mentioned embodiment.

[0081] The memory 101 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and an application required for at least one function; the data storage area may store data created according to the use of the terminal device, etc. In addition, the memory 101 may include a high-speed random access memory and may also include a non-volatile memory.

[0082] The embodiment of the present disclosure provides a product, including the above-mentioned image search testing device based on multiple scenarios.

[0083] An embodiment of the present disclosure provides a computer-readable storage medium storing computer-executable instructions, wherein the computer-executable instructions are configured to execute the above-mentioned image-searching test method based on multiple scenarios.

[0084] An embodiment of the present disclosure provides a computer program product, which includes a computer program stored on a computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer executes the above-mentioned image search test method based on multiple scenarios.

[0085] The computer-readable storage medium mentioned above may be a transient computer-readable storage medium or a non-transitory computer-readable storage medium.

[0086] The technical solution of the embodiment of the present disclosure can be embodied in the form of a software product, which is stored in a storage medium and includes one or more instructions for enabling a computer device (which may be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in the embodiment of the present disclosure. The aforementioned storage medium may be a non-transient storage medium, including: a USB flash drive, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a disk or an optical disk, and other media that can store program codes, or a transient storage medium.

[0087] The above description and the accompanying drawings fully illustrate the embodiments of the present disclosure so that those skilled in the art can practice them. Other embodiments may include structural, logical, electrical, process and other changes. The embodiments represent only possible changes. Unless explicitly required, separate components and functions are optional, and the order of operation may vary. The parts and features of some embodiments may be included in or replace the parts and features of other embodiments. Moreover, the words used in this application are only used to describe the embodiments and are not used to limit the claims. As used in the description of the embodiments and the claims, unless the context clearly indicates, the singular forms of "a", "an" and "the" are intended to include plural forms as well. Similarly, the term "and / or" as used in this application refers to any and all possible combinations of listings containing one or more associated ones. In addition, when used in the present application, the term "comprise" and its variants "comprises" and / or comprising refer to the presence of stated features, wholes, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or groups thereof. In the absence of further restrictions, the elements defined by the sentence "comprising a ..." do not exclude the presence of other identical elements in the process, method or device comprising the elements. In this article, each embodiment may focus on the differences from other embodiments, and the same and similar parts between the embodiments may refer to each other. For the methods, products, etc. disclosed in the embodiments, if they correspond to the method part disclosed in the embodiments, then the relevant parts can refer to the description of the method part.

[0088] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software may depend on the specific application and design constraints of the technical solution. The technicians may use different methods for each specific application to implement the described functions, but such implementations should not be considered to exceed the scope of the embodiments of the present disclosure. The technicians may clearly understand that, for the convenience and simplicity of description, the specific working processes of the systems, devices and units described above may refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here.

[0089] In the embodiments disclosed herein, the disclosed methods and products (including but not limited to devices, equipment, etc.) can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units can be only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between each other shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms. The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to implement this embodiment. In addition, each functional unit in the embodiment of the present disclosure may be integrated in a processing unit, or each unit may exist physically alone, or two or more units may be integrated in one unit.

[0090] The flowchart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to the embodiment of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of the code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. In some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, which can depend on the functions involved. In the description corresponding to the flowchart and the block diagram in the accompanying drawings, the operations or steps corresponding to different boxes can also occur in a different order from the order disclosed in the description, and sometimes there is no specific order between different operations or steps. For example, two consecutive operations or steps can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, which can depend on the functions involved. Each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented by a dedicated hardware-based system that performs the specified functions or actions, or may be implemented by a combination of dedicated hardware and computer instructions.

Claims

1. A multi-scenario based image search test method, characterized in that: include: Select data of the same category from the database to obtain an initial data group, wherein the number of the initial data group ranges from [500,000 to 1.5 million] groups; Selecting a search data group from the initial data group, the number of the search data groups is in the range of [20, 60] groups, wherein each search data group includes a number of pictures in the range of [10, 30], and the pictures include different angles, cameras and points; Load the data of the initial data group and the search data group into the memory through the LOAD_DATA method in the test script; After merging some convolutional layers and BN layers by improving the RepVGG algorithm and adding depthwise separable convolution to replace some convolutional layers in the RepVGG algorithm, features in the data of the initial data group and the retrieval data group are extracted; Comparing the features to obtain test results; Output the test results to generate a test report; After the improved RepVGG algorithm is used to merge some convolutional layers and BN layers and a depth-separable convolution is added to replace some convolutional layers in the RepVGG algorithm, features in the data of the initial data group and the searched data group are extracted, and the method further includes: CircleLoss is added to the RepVGG algorithm network for training. The CircleLoss formula is: Among them, s p is the intra-class similarity, s n is the out-of-class similarity, i, j, n and p are natural numbers greater than or equal to 1, L and K are the number of in-class and out-of-class targets respectively, γ is the expansion factor, and m is the relaxation factor; The comparing the features to obtain the test results comprises: The feature comparison algorithm is used to compare the features to obtain the test results. The feature comparison algorithm formula is: Among them, A and B are vector eigenvalues, λ and ρ are weight thresholds, i, j, n and p are natural numbers greater than or equal to 1, and x i and i are the coordinates of the feature points.

2. The method according to claim 1, characterized in that The depth-wise separable convolution includes depth-wise convolution and point-wise convolution. The computational complexity of the depth-wise separable convolution is: D k 2 *D p 2 *M+N*D p 2 *M(1), where D k is the size of the convolution kernel, D p Indicates that each convolution kernel slides vertically and horizontally D p times, N is the number of convolution kernels, and M is the number of channels of the convolution kernel.

3. The method according to claim 1, characterized in that Outputting the test results to generate a test report includes: Unifying the format of the test results to obtain test results in a unified format; The historical records of the test results in the unified format are backed up and a test report is generated.

4. A multi-scenario image search test device, characterized in that: The testing device is used to implement the method according to claim 1, comprising: An initial data group selection module is configured to select data of the same category from a database to obtain an initial data group, wherein the number of the initial data groups ranges from [500,000 to 1.5 million] groups; A retrieval data group selection module is configured to select a retrieval data group from the initial data group, wherein the number of the retrieval data groups ranges from [20, 60] groups, wherein each retrieval data group includes pictures in the number range of [10, 30], and the pictures include different angles, cameras and points; A loading module, configured to load the data of the initial data group and the retrieved data group into a memory through a LOAD_DATA method in a test script; A feature extraction module is configured to extract features from the data of the initial data group and the retrieval data group by improving the RepVGG algorithm to merge some convolutional layers and BN layers and adding depthwise separable convolution to replace some convolutional layers in the RepVGG algorithm; A feature comparison module is configured to compare the features to obtain a test result; The generating module is configured to output the test result and generate a test report.

5. A multi-scenario based image search test device, comprising a processor and a memory storing program instructions, characterized in that: The processor is configured to execute the multi-scenario based image search testing method as described in any one of claims 1 to 3 when running the program instructions.

6. A multi-scenario image search test product, characterized in that: It includes the image search testing device based on multiple scenarios as described in claim 4 or 5.

7. A storage medium storing program instructions, characterized in that: When the program instructions are run, they execute the multi-scenario based image search testing method as described in any one of claims 1 to 3.

Images