Method, device and equipment for identifying items in a bin, and storage medium

Abstract

The application discloses an in-warehouse article identification method, device, equipment and storage medium. The method comprises the following steps: acquiring a plurality of first identification image information of an in-warehouse article and first identification data information of an article collected by an in-warehouse RFID sensor; determining article identification data according to the first identification image information and the first identification data information; determining first article identification data through the first identification data information; determining second article identification data through the first identification image information; and taking the intersection of the first article identification data and the second article identification data as the article identification data. The application can improve the efficiency of in-warehouse article identification, achieve the goal of accurate identification and rapid identification, and effectively prevent the problem of article identification error.

Description

Technical Field

[0001] This disclosure generally relates to the field of image recognition technology, and specifically to a method, apparatus, equipment, and storage medium for identifying items inside a warehouse. Background Technology

[0002] With the advancement of automation technology in medical equipment, automated storage, identification, and distribution of some medical devices have become possible. However, in the current technology, the storage, identification, and distribution of medical devices still mainly rely on manual labor. For example, setting QR codes or RFID tags on medical devices, or using visual recognition technology through convolutional neural networks, or using some sensor devices, are all relatively complex and difficult to achieve effective management of medical supplies. In particular, for some medical supply storage warehouses, accurate identification of the materials in the warehouse can improve the efficiency of material management. Summary of the Invention

[0003] In view of the above-mentioned defects or deficiencies in the prior art, it is desirable to provide a method, apparatus, device and storage medium for identifying items in a warehouse that can meet the needs of the art.

[0004] Based on one aspect of the embodiments of the present invention, this application provides a method for identifying items in a warehouse, the method comprising:

[0005] Acquire multiple first identification image information of items in the warehouse, as well as first identification data information of items collected by RFID sensors in the warehouse. The first identification image information is image information of items in the warehouse collected from multiple angles by an image acquisition device.

[0006] Based on the first identified image information and the first identified data information, the item identification data is determined;

[0007] The step of determining the item identification data based on the first identification image information and the first identification data information includes:

[0008] The first item identification data is determined using the first identification data information;

[0009] The identification data of the second item is determined using the first identified image information;

[0010] The intersection of the first item identification data and the second item identification data is taken as the item identification data.

[0011] In another embodiment of this application, the method further includes:

[0012] Obtain quantity and quality information of items in the warehouse;

[0013] Based on the quantity and quality information of the items, obtain third item identification data;

[0014] Based on the first item identification data, the second item identification data, and the third item identification data, the item identification data is determined.

[0015] In another embodiment of this application, determining the item identification data based on the first item identification data, the second item identification data, and the third item identification data includes:

[0016] Find the intersection of the first item identification data and the third item identification data;

[0017] The intersection of the given data and the second item identification data is used to obtain the item identification data.

[0018] In another embodiment of this application, determining the second item identification data through the first identification image information includes:

[0019] The first recognition image information is subjected to a first processing to obtain a first processed image, wherein the first processing is a correction processing of the first recognition image information;

[0020] The first processed image is subjected to a second processing to obtain a second processed image, wherein the second processing is image enhancement processing of the first processed image;

[0021] The second processed image is compared with image sample data in the image library, and a similarity threshold between the second processed image and the image sample data is set to obtain the second item recognition data.

[0022] In another embodiment of this application, the step of comparing the second processed image with image sample data in the image library, setting a similarity threshold between the second processed image and the image sample data, and obtaining second item recognition data includes:

[0023] The image feature information of the second processed image is compared with the image feature information of each image sample data in the image sample data in the image library for similarity. A first target image sample whose similarity meets the first set condition is selected from the image sample data in the image library, and a second target image sample whose similarity meets the second set condition is selected from the image sample data in the image library.

[0024] The difference in image structure features between the average image structure features of the second target image sample and the average image structure features of the first target image sample is determined; and the difference in image chromaticity features between the average image chromaticity features of the second target image sample and the average image chromaticity features of the first target image sample is determined.

[0025] Based on the differences in image structure features and the differences in image color features, the second processed image is subjected to image structure processing and image color processing to obtain a third processed image and a fourth processed image that are adapted to the first target image sample and the second target image sample, respectively.

[0026] The similarity between the third processed image and the first target sample image, and between the fourth processed image and the second target sample image are compared respectively, and the second item recognition data is obtained according to the set similarity threshold.

[0027] In another embodiment of this application, the step of selecting a first target image sample whose similarity meets a first predetermined condition from the image sample data in the image library, and selecting a second target image sample whose similarity meets a second predetermined condition from the image sample data in the image library, includes:

[0028] The descriptive features of the second processed image are compared with the descriptive features of each sample. Candidate first target sample images that meet the first sub-condition are selected from the image sample data in the image library according to the order of similarity from high to low. Candidate second target image samples that meet the second sub-condition are selected from the image sample data in the image library. The first sub-condition is the front view of the image and the second sub-condition is the top view of the image.

[0029] The key point information of the second processed image is matched with the key point similarity of each image sample in the candidate first target sample image and the candidate second target image sample. The first target sample image with key point similarity satisfying the set threshold is selected from the candidate first target sample image according to the order of key point similarity from high to low. The second target image sample with key point similarity satisfying the set threshold is selected from the candidate second target image sample.

[0030] In another embodiment of this application, the first processing of the first recognized image information to obtain a first processed image, wherein the first processing is a correction processing of the first recognized image information, includes:

[0031] The adjacent images in the first identified image information are corrected. The adjacent images are two adjacent images acquired by the same image acquisition device. The steps of correcting the adjacent images include:

[0032] Obtain any frame of the first recognized image information as the base image;

[0033] Calculate the similarity value between the central image patch in the base image and any image patch in the neighboring images of the base image:

[0034] ;

[0035] in, The pixel mean of the k-th image block among the neighboring images of the base image. The pixel mean of the base image patch. It is the average pixel value of any image block in the neighboring images of the base image.

[0036] Similarity value The highest corresponding image block As the central image block Matching image patches in adjacent images;

[0037] Extract separately and SIFT feature points are obtained in 8 directions, and the feature points are matched to obtain the matching set of feature points as { },in, For a set of matched SIFT feature points, Based on image At the SIFT feature point in direction 1, Neighboring images of the base image In and basic images of Construct a feature point matching matrix by matching the corresponding feature points:

[0038] ;

[0039] Based on the relationship between the feature point matching matrix and the basic processing matrix F, the feature point matching matrix is ​​decomposed into singular values. The singular matrix corresponding to the smallest singular value after decomposition is the solution of the basic processing matrix F.

[0040] Based on the solution of the fundamental processing matrix F, we obtain The extreme point;

[0041] Perform matrix transformations on neighboring images of the base image, and simultaneously convert the disparity of the base image into depth values;

[0042] The transformed image is used as the corrected imaging result, and the imaging result of the next adjacent image is corrected by rectangular transformation using the corrected base image to obtain the first processed image.

[0043] In another embodiment of this application, the second processing of the first processed image to obtain a second processed image, wherein the second processing is image enhancement processing of the first processed image, includes:

[0044] Each image in the first processed image is subjected to gamma enhancement processing, wherein the gamma enhancement processing is as follows:

[0045] ;

[0046] In the formula, One of the images in the second processed image, Given the gamma parameter, the image sequence for the second processed image is as follows:

[0047] .

[0048] According to another aspect of the present invention, a warehouse item identification device is disclosed, the device comprising:

[0049] The acquisition module is used to acquire multiple first identification image information of items in the warehouse, as well as first identification data information of items collected by RFID sensors in the warehouse. The first identification image information is image information of items in the warehouse collected from multiple angles by an image acquisition device.

[0050] The identification module is used to determine item identification data based on the first identification image information and the first identification data information; wherein, determining the item identification data based on the first identification image information and the first identification data information includes: determining first item identification data through the first identification data information; determining second item identification data through the first identification image information; and taking the intersection of the first item identification data and the second item identification data as the item identification data.

[0051] According to another aspect of the present invention, an electronic device is disclosed, the electronic device including one or more processors and a memory, the memory being used to store one or more programs; when the one or more programs are executed by the processor, the processor enables the processor to implement the warehouse item identification method provided in the various embodiments of the present invention.

[0052] According to another aspect of the present invention, a computer-readable storage medium storing a computer program is disclosed, which, when executed, implements the warehouse item identification method provided in various embodiments of the present invention.

[0053] In this embodiment, multiple first identification image information of items in the warehouse and first identification data information of items collected by RFID sensors in the warehouse are acquired; item identification data is determined based on the first identification image information and the first identification data information; first item identification data is determined based on the first identification data information; second item identification data is determined based on the first identification image information; and the intersection of the first item identification data and the second item identification data is taken as the item identification data. This application can improve the efficiency of item identification in the warehouse, achieve the goal of accurate and rapid identification, and effectively prevent the problem of incorrect product identification. Attached Figure Description

[0054] Other features, objects, and advantages of this application will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0055] Figure 1 This is a flowchart of a warehouse item identification method provided in one embodiment of this application;

[0056] Figure 2 This is a schematic diagram of the structure of an in-warehouse item identification device provided in one embodiment of this application;

[0057] Figure 3 This is an internal structural diagram of an electronic device provided in one embodiment of this application. Detailed Implementation

[0058] The present application will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, only the parts relevant to the invention are shown in the accompanying drawings.

[0059] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0060] Please refer to Figure 1 It illustrates an exemplary flow of the warehouse item identification method that can be applied to embodiments of this application.

[0061] like Figure 1 As shown, in step 110, multiple first identification image information of items in the warehouse and first identification data information of items collected by RFID sensors in the warehouse are obtained. The first identification image information is image information of items in the warehouse collected from multiple angles by image acquisition devices.

[0062] Specifically, by installing multiple cameras and other image acquisition devices within the storage compartment, image information of the items is captured from multiple angles. In the embodiments of this application, the main captured images of the items are front view images and top view images. The front view image provides information about the front of the item, allowing for a preliminary determination of its category. The top view image provides information about the item from above, combined with the front view... Figure 1 To obtain more accurate information about items, RFID sensors are installed in the storage compartment, and RFID tags are attached to the items. It's important to note that since RFID tags are manually set, the actual item is not verified against the tag during application. Therefore, RFID tags alone cannot accurately retrieve information about the actual item; they only provide the information stored within the tag. Thus, a combination of image recognition and RFID tags is necessary for more accurate information. In practice, a single RFID tag may store information about multiple items that are related or similar, such as having similar usage, being stored in similar locations, or being stored in the same warehouse. When these items are stored in a single compartment, the RFID sensors will identify all the information stored in that RFID tag, requiring further verification.

[0063] Specifically, after collecting image information of items in the warehouse, the images of the items collected by the RFID sensors in the warehouse are uniformly set as the first identification images, and the image information of the items collected by the RFID sensors in the warehouse is set as the first identification data.

[0064] In step 120, the item identification data is determined based on the first identification image information and the first identification data information.

[0065] Specifically, the first identification data information can be used to obtain the item information stored in the RFID tag of the item. The first identification image information can be used to determine which information stored in the first identification data information is accurate based on the item image. In this way, an RFID tag can store multiple types of item information. Thus, by combining the first identification image information and the first identification data information, the item identification data can be determined.

[0066] Specifically, in one embodiment of this application, determining the item identification data based on the first identification image information and the first identification data information includes:

[0067] The first item identification data is determined by the first identification data information. Specifically, since the first identification data information may contain multiple product information, for example, if the item stored in a storage warehouse is shoes, then the RFID tag corresponding to the first identification data information will store information on all types of shoes in the storage warehouse, because it allows these shoes to be stored in the storage warehouse. Therefore, the RFID sensor in the warehouse may only be able to identify that the item is shoes and that this type of shoes is allowed to be stored in the storage warehouse. Moreover, all the information of all types of shoes stored in the RFID tag can be read. As for the specific type of shoes, it cannot be completely identified.

[0068] The first identification image information is used to determine the second item identification data. Specifically, the appearance of the item is further identified using the first identification image information. The image recognition method processes the image according to a corresponding algorithm, and then compares the processed image with the images in the image sample data to obtain the item information reflected by the image. It should be noted that the first item identification data determined by the first identification image information cannot completely reflect the actual item information, as it is affected by the quality, angle, and image processing algorithm of the image acquired by the image acquisition device.

[0069] The intersection of the first item identification data and the second item identification data is taken as the item identification data.

[0070] Specifically, the image information obtained from the first item recognition data and the second item recognition data is filtered, and the item data identified by the two are intersected to obtain the item recognition data information that satisfies the condition of being present in both the first item recognition data and the second item recognition data.

[0071] Specifically, in one embodiment of this application, the warehouse item identification method further includes: obtaining quantity information and quality information of items in the warehouse; specifically, since different items have different qualities, the quality status of a single item can be obtained by knowing the quality and quantity of the items. At the same time, since the quality information of the items is also stored on the RFID tag of the items, the quality status of the items can also be used to further determine the item data.

[0072] Based on the quantity and quality information of the items, third item identification data is obtained; specifically, the third item identification data is the quality information of a unit item, such as the quality information of a single item, or the quality information of a combination of multiple items combined in a set manner.

[0073] Based on the first item identification data, the second item identification data, and the third item identification data, the item identification data is determined.

[0074] Specifically, by using the first item identification data, the second item identification data, and the third item identification data together to identify the acquired item information, more accurate item identification data can be obtained.

[0075] Specifically, in one embodiment of this application, determining the item identification data based on the first item identification data, the second item identification data, and the third item identification data includes:

[0076] The intersection of the first item identification data and the third item identification data is calculated. Specifically, since the first item identification data is based on the data read from the RFID tag of the item, and the third item identification data is the mass data of a unit item, the intersection of the two can further limit the scope of item identification. That is, it needs to simultaneously satisfy the condition that the mass data of a unit item is consistent with the mass data of the item identified by the RFID tag. It should be noted that the intersection of the two may not necessarily uniquely identify a certain item information, because there may be cases where the mass of a unit item is the same. For example, a storage warehouse stores various types of shoes. The RFID tag can identify the type of the item as shoes, and all the shoe information in the storage warehouse can be identified by the RFID tag. Then, if two or more types of shoes have the same mass, then the intersection of the first item identification data and the third item identification data will identify these two or more types of shoes that meet the condition of mass.

[0077] The intersection of the first and third item recognition data is then intersected with the second item recognition data to obtain the item recognition data. Specifically, the item recognition data obtained by intersecting the first and third item recognition data is then intersected with the second item recognition data obtained through the first recognition image information. This further refines the identification of items, resulting in more accurate item recognition.

[0078] Specifically, in one embodiment of this application, determining the second item identification data through the first identification image information includes:

[0079] The first recognition image information is subjected to a first processing to obtain a first processed image, wherein the first processing is a correction processing of the first recognition image information;

[0080] The first processed image is subjected to a second processing to obtain a second processed image, wherein the second processing is image enhancement processing of the first processed image;

[0081] The second processed image is compared with image sample data in the image library, and a similarity threshold between the second processed image and the image sample data is set to obtain the second item recognition data.

[0082] Specifically, in one embodiment of this application, the step of comparing the second processed image with image sample data in the image library, setting a similarity threshold between the second processed image and the image sample data, and obtaining second item recognition data includes:

[0083] The image feature information of the second processed image is compared with the image feature information of each image sample data in the image sample data of the image library for similarity. A first target image sample whose similarity meets a first set condition is selected from the image sample data of the image library, and a second target image sample whose similarity meets a second set condition is selected from the image sample data of the image library. Specifically, by comparing the image feature information of the second processed image with the image feature information of each image sample data in the image sample data of the image library, a first target image sample whose similarity meets the first set condition can be selected from the image sample data of the image library. Here, the first target image sample is a front view image sample, and a second target image sample whose similarity meets the second set condition can be selected from the image sample data of the image library. Here, the second target image sample is a top view image sample.

[0084] The first setting condition can be that the similarity is greater than a first similarity threshold, so that the image sample of the main view can be selected by using similarity as a comparison threshold; or, the first setting condition can be that a first predetermined number of image samples of the main view are selected as the first target image samples in descending order of similarity, starting from the highest similarity, so that similarity and quantity are combined as constraints at the same time.

[0085] Similar to the first setting condition, the second setting condition can be that the similarity is greater than the second similarity threshold, so that the second target image sample can be selected by using similarity as a comparison threshold; or, the second setting condition can be that a second predetermined number of top view samples are selected as the second target image samples in descending order of similarity, starting from the highest similarity, so that similarity and quantity are combined as restrictions at the same time.

[0086] Regarding the first setting condition and the second setting condition, the first similarity threshold and the second similarity threshold may be equal or unequal, and the first predetermined quantity and the second predetermined quantity may be equal or unequal. That is to say, the selection of the first target image sample and the second target image sample may be set with exactly the same filtering conditions, or different filtering conditions may be set respectively. This disclosure embodiment does not limit this.

[0087] The difference in image structure features between the average image structure features of the second target image sample and the average image structure features of the first target image sample is determined; and the difference in image chromaticity features between the average image chromaticity features of the second target image sample and the average image chromaticity features of the first target image sample is determined.

[0088] Specifically, image structural features are structural parameters used to describe the entire image. In practice, for example, image structural features and image chromaticity features of each image sample can be extracted using a VGG network (a type of neural convolutional network). The image structural features and image chromaticity features extracted by the VGG network can be represented as VGG features. Thus, the average image structural features and average image chromaticity features of all image samples in the first target image sample, as well as the average image structural features and average image chromaticity features of all image samples in the second target image sample, can be obtained. Furthermore, the difference in image structural features between the average image structural features of the first target image sample and the average image structural features of the second target image sample can be obtained, as well as the difference in image chromaticity features between the average image chromaticity features of the first target image sample and the average image chromaticity features of the second target image sample.

[0089] Based on the differences in image structural features and image chromaticity features, the second processed image is subjected to image structural processing and image chromaticity processing to obtain a third processed image and a fourth processed image that are adapted to the first target image sample and the second target image sample, respectively. Specifically, the processing parameters of the image acquired by the image acquisition device can be obtained through the parameters of the differences in image structural features and image chromaticity features. According to the differences in image structural features, the second processed image is structurally transformed to achieve structural parameters that are adapted to the image samples in the first target image sample and the second target image sample. By performing image chromaticity processing on the second processed image, chromaticity parameters that are adapted to the image samples in the first target image sample and the second target image sample are obtained, so as to better complete the identification of the object through the first target image sample and the second target image sample. The third processed image and the fourth processed image are the front view and top view images of the second processed image after sequentially performing image structural processing and image chromaticity processing.

[0090] The similarity between the third processed image and the first target sample image, and between the fourth processed image and the second target sample image, are compared respectively. Based on a set similarity threshold, the second item recognition data is obtained. Specifically, since the third processed image and the fourth processed image are the front view and top view images of the second processed image after image structure processing and image color processing, the third processed image can be recognized and adapted to the first target image sample, and the fourth processed image can be recognized and adapted to the second target image sample.

[0091] The step of selecting a first target image sample whose similarity meets a first predetermined condition from the image sample data in the image library, and selecting a second target image sample whose similarity meets a second predetermined condition from the image sample data in the image library, includes:

[0092] The descriptive features of the second processed image are compared with the descriptive features of each sample. Candidate first target sample images that meet the first sub-condition are selected from the image sample data in the image library according to the order of similarity from high to low. Candidate second target image samples that meet the second sub-condition are selected from the image sample data in the image library. The first sub-condition is the front view of the image and the second sub-condition is the top view of the image.

[0093] The key point information of the second processed image is matched with the key point similarity of each image sample in the candidate first target sample image and the candidate second target image sample. The first target sample image with key point similarity satisfying the set threshold is selected from the candidate first target sample image according to the order of key point similarity from high to low. The second target image sample with key point similarity satisfying the set threshold is selected from the candidate second target image sample.

[0094] Specifically, in one embodiment of this application, the first processing of the first recognized image information to obtain a first processed image, wherein the first processing is a correction processing of the first recognized image information, includes:

[0095] The adjacent images in the first identified image information are corrected. The adjacent images are two adjacent images acquired by the same image acquisition device. The steps of correcting the adjacent images include:

[0096] Obtain any frame of the first recognized image information as the base image;

[0097] Calculate the similarity value between the central image patch in the base image and any image patch in the neighboring images of the base image:

[0098] ;

[0099] in, The pixel mean of the k-th image block among the neighboring images of the base image. The pixel mean of the base image patch. It is the average pixel value of any image block in the neighboring images of the base image.

[0100] Similarity value The highest corresponding image block As the central image block Matching image patches in adjacent images;

[0101] Extract separately and SIFT feature points are obtained in 8 directions, and the feature points are matched to obtain the matching set of feature points as { },in, For a set of matched SIFT feature points, Basic Image At the SIFT feature point in direction 1, Neighboring images of the base image In and basic images of Construct a feature point matching matrix by matching the corresponding feature points:

[0102] ;

[0103] Based on the relationship between the feature point matching matrix and the basic processing matrix F, the feature point matching matrix is ​​decomposed into singular values. The singular matrix corresponding to the smallest singular value after decomposition is the solution of the basic processing matrix F.

[0104] Based on the solution of the fundamental processing matrix F, we obtain The extreme point;

[0105] Perform matrix transformations on neighboring images of the base image, and simultaneously convert the disparity of the base image into depth values;

[0106] The transformed image is used as the corrected imaging result, and the imaging result of the next adjacent image is corrected by rectangular transformation using the corrected base image to obtain the first processed image.

[0107] Specifically, in one embodiment of this application, the second processing of the first processed image to obtain a second processed image, wherein the second processing is image enhancement processing of the first processed image, includes:

[0108] Each image in the first processed image is subjected to gamma enhancement processing, wherein the gamma enhancement processing is as follows:

[0109] ;

[0110] In the formula, One of the images in the second processed image, Given the gamma parameter, the image sequence for the second processed image is as follows:

[0111] .

[0112] In the formula, The second processed image is the result of enhancing the first processed image.

[0113] In this embodiment, multiple first identification image information of items in the warehouse and first identification data information of items collected by RFID sensors in the warehouse are acquired; item identification data is determined based on the first identification image information and the first identification data information; first item identification data is determined based on the first identification data information; second item identification data is determined based on the first identification image information; and the intersection of the first item identification data and the second item identification data is taken as the item identification data. This application can improve the efficiency of item identification in the warehouse, achieve the goal of accurate and rapid identification, and effectively prevent the problem of incorrect product identification.

[0114] It should be understood that, although Figure 1 The steps in the flowchart are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order in which these steps are executed, and they can be performed in other orders. Figure 1 At least some of the steps in the process may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least some of the sub-steps or stages of other steps.

[0115] Figure 2 This is a schematic diagram of the structure of an in-warehouse item identification device provided in one embodiment of this application, as shown below. Figure 2 As shown, the in-warehouse item identification device includes:

[0116] Acquisition module, recognition module;

[0117] The acquisition module is used to acquire multiple first identification image information of items in the warehouse, as well as first identification data information of items collected by RFID sensors in the warehouse. The first identification image information is image information of items in the warehouse collected from multiple angles by an image acquisition device.

[0118] The identification module is used to determine item identification data based on the first identification image information and the first identification data information; wherein, determining the item identification data based on the first identification image information and the first identification data information includes: determining first item identification data through the first identification data information; determining second item identification data through the first identification image information; and taking the intersection of the first item identification data and the second item identification data as the item identification data.

[0119] Specifically, in another embodiment of this application, the acquisition module is used to acquire quantity information and quality information of items in the warehouse; based on the quantity information and quality information of the items, acquire third item identification data; the identification module is used to determine item identification data based on the first item identification data, the second item identification data and the third item identification data.

[0120] Specifically, in another embodiment of this application, the identification module is used to find the intersection of the first item identification data and the third item identification data; and to find the intersection of the intersection with the second item identification data to obtain item identification data.

[0121] Specifically, in another embodiment of this application, the recognition module is used to perform a first processing on the first recognition image information to obtain a first processed image, wherein the first processing is to perform correction processing on the first recognition image information; perform a second processing on the first processed image to obtain a second processed image, wherein the second processing is to perform image enhancement processing on the first processed image; compare the second processed image with image sample data in the image library, set a similarity threshold between the second processed image and the image sample data, and obtain second item recognition data.

[0122] Specifically, in another embodiment of this application, the recognition module is used to compare the image feature information of the second processed image with the image feature information of each image sample data in the image sample data in the image library, and select a first target image sample whose similarity meets a first set condition from the image sample data in the image library, and select a second target image sample whose similarity meets a second set condition from the image sample data in the image library; determine the image structure feature difference between the average image structure feature of the second target image sample and the average image structure feature of the first target image sample, and determine the image chroma feature difference between the average image chroma feature of the second target image sample and the average image chroma feature of the first target image sample; perform image structure processing and image chroma processing on the second processed image according to the image structure feature difference and the image chroma feature difference, respectively obtaining a third processed image and a fourth processed image of the second processed image that are adapted to the first target image sample and the second target image sample; compare the similarity between the third processed image and the first target sample image, and the fourth processed image and the second target sample image, respectively, and obtain second item recognition data according to the set similarity threshold.

[0123] Specifically, in another embodiment of this application, the recognition module is used to compare the descriptive features of the second processed image with the descriptive features of each sample, and to select candidate first target sample images whose similarity satisfies a first sub-condition from the image sample data in the image library in descending order of similarity, and to select candidate second target image samples whose similarity satisfies a second sub-condition from the image sample data in the image library, wherein the first sub-condition is a front view of the image and the second sub-condition is a top view of the image; to match the key point information of the second processed image with the key point similarity of each image sample in the candidate first target sample image and the candidate second target image sample, and to select first target sample images whose key point similarity satisfies a set threshold from the candidate first target sample images in descending order of key point similarity, and to select second target image samples whose key point similarity satisfies a set threshold from the candidate second target image samples.

[0124] Specifically, in another embodiment of this application, the recognition module is used to perform correction processing on adjacent images in the first recognition image information. The adjacent images are two adjacent images acquired by the same image acquisition device. The step of performing correction processing on the adjacent images includes: obtaining any frame image in the first recognition image information as a base image; calculating the similarity value between the center image block in the base image and any image block in the adjacent images of the base image.

[0125] ;

[0126] in, The pixel mean of the k-th image block among the neighboring images of the base image. The pixel mean of the base image patch. The similarity value is the pixel mean of any image patch in the neighboring images of the base image; The highest corresponding image block As the central image block Matching image patches in adjacent images; extracting them respectively. and SIFT feature points are obtained in 8 directions, and the feature points are matched to obtain the matching set of feature points as { },in, For a set of matched SIFT feature points, Basic Image At the SIFT feature point in direction 1, Neighboring images of the base image In and basic images of Construct a feature point matching matrix by matching the corresponding feature points:

[0127] ;

[0128] Based on the relationship between the feature point matching matrix and the fundamental processing matrix F, singular value decomposition is performed on the feature point matching matrix. The singular matrix corresponding to the smallest singular value after decomposition is the solution to the fundamental processing matrix F. Based on the solution to the fundamental processing matrix F, we obtain... The extreme points of the base image are determined; a matrix transformation is performed on the adjacent images of the base image, and the disparity of the base image is converted into a depth value; the transformed image is used as the corrected imaging result, and the imaging result of the next adjacent image of the corrected base image is subjected to rectangular transformation correction to obtain the first processed image.

[0129] Specifically, in another embodiment of this application, the recognition module is used to perform gamma enhancement processing on each image in the first processed image, wherein the gamma enhancement processing is as follows:

[0130] In the formula, One of the images in the second processed image, Given the gamma parameter, the image sequence for the second processed image is as follows:

[0131] .

[0132] In this embodiment, the acquisition module acquires multiple first identification image information of items in the warehouse, as well as first identification data information of items collected by RFID sensors in the warehouse. The first identification image information consists of images of items in the warehouse collected from multiple angles by an image acquisition device. The identification module determines item identification data based on the first identification image information and the first identification data information. Specifically, determining item identification data based on the first identification image information and the first identification data information includes: determining first item identification data using the first identification data information; determining second item identification data using the first identification image information; and using the intersection of the first item identification data and the second item identification data as the item identification data. This application can improve the efficiency of item identification in the warehouse, achieve the goals of accurate and rapid identification, and effectively prevent product identification errors.

[0133] Specific limitations regarding the in-warehouse item identification device can be found in the limitations of the in-warehouse item identification method described above, and will not be repeated here. Each module in the aforementioned in-warehouse item identification device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the corresponding operations of each module.

[0134] Specifically, according to embodiments of this disclosure, such as Figure 3 As shown, the present invention discloses an electronic device, which includes one or more processors and a memory. The memory is used to store one or more programs. When the one or more programs are executed by the processor, the processor implements the in-warehouse item identification method described in the embodiments of the present invention.

[0135] In particular, according to embodiments of this disclosure, the warehouse item identification method described in any of the above embodiments can be implemented as a computer software program. For example, embodiments of this disclosure include a computer program product comprising a computer program tangibly embodied on a machine-readable medium, the computer program containing program code for performing the warehouse item identification method. In such embodiments, the computer program can be downloaded and installed from a network via a communication component, and / or installed from a removable medium.

[0136] The one or more programs stored in the read-only memory (ROM) or the random access memory (RAM) perform various appropriate actions and processes. The RAM includes software programs for the server to complete corresponding business operations, as well as various programs and data required for vehicle driving operations. The server, its controlled hardware devices, the ROM, and the RAM are interconnected via a bus, and various input / output interfaces are also connected to the bus.

[0137] The following components are connected to the input / output interface: input sections including keyboards, mice, etc.; output sections including cathode ray tube (CRT) displays, liquid crystal displays (LCDs), and speakers; and communication sections including network interface cards such as LAN cards and modems. The communication section performs communication processing via a network such as the Internet. Drives are also connected to the input / output interface as needed. Removable media, such as disks, optical discs, magneto-optical discs, semiconductor memory, etc., are installed on the drive as needed so that computer programs read from them can be installed into memory as required.

[0138] In particular, according to embodiments of this disclosure, the warehouse item identification method described in any of the above embodiments can be implemented as a computer software program. For example, embodiments of this disclosure include a computer program product comprising a computer program tangibly embodied on a machine-readable medium, the computer program containing program code for performing the warehouse item identification method. In such embodiments, the computer program can be downloaded and installed from a network via a communication component, and / or installed from a removable medium.

[0139] The units or modules described in the embodiments of this application can be implemented in software or hardware. The described units or modules can also be located in a processor. The names of these units or modules do not necessarily constitute a limitation on the unit or module itself.

[0140] The above description is merely a preferred embodiment of this application and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in this application is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the inventive concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features with similar functions disclosed in this application.

Claims

1. A method for identifying items inside a warehouse, characterized in that, The method includes: Acquire multiple first identification image information of items in the warehouse, as well as first identification data information of items collected by RFID sensors in the warehouse. The first identification image information is image information of items in the warehouse collected from multiple angles by an image acquisition device. Based on the first identified image information and the first identified data information, the item identification data is determined; The step of determining the item identification data based on the first identification image information and the first identification data information includes: The first item identification data is determined using the first identification data information; The identification data of the second item is determined using the first identified image information; The intersection of the first item identification data and the second item identification data is taken as the item identification data; The step of determining the second item identification data through the first identification image information includes: The first recognition image information is subjected to a first processing to obtain a first processed image, wherein the first processing is a correction processing of the first recognition image information; The first processed image is subjected to a second processing to obtain a second processed image, wherein the second processing is image enhancement processing of the first processed image; The second processed image is compared with image sample data in the image library, and a similarity threshold between the second processed image and the image sample data is set to obtain the second item recognition data. The first processing of the first recognized image information to obtain a first processed image, wherein the first processing is a correction process on the first recognized image information, includes: The adjacent images in the first identified image information are corrected. The adjacent images are two adjacent images acquired by the same image acquisition device. The steps of correcting the adjacent images include: Obtain any frame of the first recognized image information as the base image; Calculate the similarity value between the central image patch in the base image and any image patch in the neighboring images of the base image: ; in, The pixel mean of the k-th image block among the neighboring images of the base image. The pixel mean of the base image patch. It is the average pixel value of any image block in the neighboring images of the base image; Similarity value The highest corresponding image block As the central image block Matching image patches in adjacent images; Extract separately and SIFT feature points are obtained in 8 directions, and the feature points are matched to obtain the matching set of feature points as { },in, For a set of matched SIFT feature points, Basic Image At the SIFT feature point in direction 1, Neighboring images of the base image In and basic images of Construct a feature point matching matrix by matching the corresponding feature points: ； Based on the relationship between the feature point matching matrix and the basic processing matrix F, the feature point matching matrix is ​​decomposed into singular values. The singular matrix corresponding to the smallest singular value after decomposition is the solution of the basic processing matrix F. Based on the solution of the fundamental processing matrix F, we obtain The extreme point; Perform matrix transformations on neighboring images of the base image, and simultaneously convert the disparity of the base image into depth values; The transformed image is used as the corrected imaging result, and the imaging result of the next adjacent image is corrected by rectangular transformation using the corrected base image to obtain the first processed image.

2. The method according to claim 1, characterized in that, The method further includes: Obtain quantity and quality information of items in the warehouse; Based on the quantity and quality information of the items, obtain third item identification data; Based on the first item identification data, the second item identification data, and the third item identification data, the item identification data is determined.

3. The method according to claim 2, characterized in that, The step of determining the item identification data based on the first item identification data, the second item identification data, and the third item identification data includes: Find the intersection of the first item identification data and the third item identification data; The intersection of the given data and the second item identification data is used to obtain the item identification data.

4. The method according to claim 1, characterized in that, The step of comparing the second processed image with image sample data in the image library, setting a similarity threshold between the second processed image and the image sample data, and obtaining second item recognition data includes: The image feature information of the second processed image is compared with the image feature information of each image sample data in the image sample data in the image library for similarity. A first target image sample whose similarity meets the first set condition is selected from the image sample data in the image library, and a second target image sample whose similarity meets the second set condition is selected from the image sample data in the image library. The difference in image structure features between the average image structure features of the second target image sample and the average image structure features of the first target image sample is determined; and the difference in image chromaticity features between the average image chromaticity features of the second target image sample and the average image chromaticity features of the first target image sample is determined. Based on the differences in image structure features and the differences in image color features, the second processed image is subjected to image structure processing and image color processing to obtain a third processed image and a fourth processed image that are adapted to the first target image sample and the second target image sample, respectively. The similarity between the third processed image and the first target sample image, and between the fourth processed image and the second target sample image are compared respectively, and the second item recognition data is obtained according to the set similarity threshold.

5. The method according to claim 4, characterized in that, The step of selecting a first target image sample whose similarity meets a first predetermined condition from the image sample data in the image library, and selecting a second target image sample whose similarity meets a second predetermined condition from the image sample data in the image library, includes: The descriptive features of the second processed image are compared with the descriptive features of each sample. Candidate first target sample images that meet the first sub-condition are selected from the image sample data in the image library according to the order of similarity from high to low. Candidate second target image samples that meet the second sub-condition are selected from the image sample data in the image library. The first sub-condition is the front view of the image and the second sub-condition is the top view of the image. The key point information of the second processed image is matched with the key point similarity of each image sample in the candidate first target sample image and the candidate second target image sample. The first target sample image with key point similarity satisfying the set threshold is selected from the candidate first target sample image according to the order of key point similarity from high to low. The second target image sample with key point similarity satisfying the set threshold is selected from the candidate second target image sample.

6. The method according to claim 1, characterized in that, The second processing of the first processed image to obtain a second processed image, wherein the second processing is image enhancement processing of the first processed image, includes: Each image in the first processed image is subjected to gamma enhancement processing, wherein the gamma enhancement processing is as follows: ； In the formula, One of the images in the second processed image, Given the gamma parameter, the image sequence for the second processed image is as follows: 。 7. A warehouse item identification device, characterized in that, The device includes: The acquisition module is used to acquire multiple first identification image information of items in the warehouse, as well as first identification data information of items collected by RFID sensors in the warehouse. The first identification image information is image information of items in the warehouse collected from multiple angles by an image acquisition device. The identification module is used to determine item identification data based on the first identification image information and the first identification data information; wherein, determining the item identification data based on the first identification image information and the first identification data information includes: determining first item identification data through the first identification data information; determining second item identification data through the first identification image information; and taking the intersection of the first item identification data and the second item identification data as the item identification data; The first processing is used to perform a first processing on the first recognition image information to obtain a first processed image, wherein the first processing is to perform a correction processing on the first recognition image information. The first processed image is subjected to a second processing to obtain a second processed image, wherein the second processing is image enhancement processing of the first processed image; The second processed image is compared with image sample data in the image library, and a similarity threshold between the second processed image and the image sample data is set to obtain the second item recognition data. The method for correcting adjacent images in the first identified image information, wherein the adjacent images are two adjacent images acquired by the same image acquisition device, includes the following steps: Obtain any frame of the first recognized image information as the base image; Calculate the similarity value between the central image patch in the base image and any image patch in the neighboring images of the base image: ; in, The pixel mean of the k-th image block among the neighboring images of the base image. The pixel mean of the base image patch. It is the average pixel value of any image block in the neighboring images of the base image; Similarity value The highest corresponding image block As the central image block Matching image patches in adjacent images; Extract separately and SIFT feature points are obtained in 8 directions, and the feature points are matched to obtain the matching set of feature points as { },in, For a set of matched SIFT feature points, Basic Image At the SIFT feature point in direction 1, Neighboring images of the base image In and basic images of Construct a feature point matching matrix by matching the corresponding feature points: ； Based on the relationship between the feature point matching matrix and the basic processing matrix F, the feature point matching matrix is ​​decomposed into singular values. The singular matrix corresponding to the smallest singular value after decomposition is the solution of the basic processing matrix F. Based on the solution of the fundamental processing matrix F, we obtain The extreme point; Perform matrix transformations on neighboring images of the base image, and simultaneously convert the disparity of the base image into depth values; The transformed image is used as the corrected imaging result, and the imaging result of the next adjacent image is corrected by rectangular transformation using the corrected base image to obtain the first processed image.

8. An electronic device, characterized in that, The device includes one or more processors and a memory, the memory being used to store one or more programs; When the processor executes the one or more programs, the processor performs the method as described in any one of claims 1 to 6.

Images