Information processing method, apparatus and device

Abstract

Embodiments of the present application provide an information processing method, device and equipment. Wherein, a shelf picture collected by a mobile robot in a target site is obtained; at least one shelf is deployed in the target site, the at least one shelf respectively displays at least one article and is provided with a first information carrier at a respective display position of the at least one article, the first information carrier carries article information of the corresponding article; the first information carrier and the article information carried by the first information carrier in the shelf picture are identified; and based on the article information of at least one article belonging to the same shelf, shelf display information is generated. The technical scheme provided by the embodiments of the present application realizes efficient and accurate information acquisition.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the technical field of computer application, and particularly relate to an information processing method, device and equipment. BACKGROUND

[0002] At present, some large supermarkets, convenience stores, warehouses and warehouse-style shopping malls and the like offline places will usually use shelves to display and manage goods.

[0003] In order to facilitate goods management and place operation and the like, it is usually necessary to understand the display situation of goods on the shelves. Since the goods on the shelves will change frequently, the use of manual recording method will consume a large amount of manpower and time cost, and the accuracy is not high, therefore, an efficient and accurate processing method is needed to realize the acquisition of shelf display information. SUMMARY

[0004] Embodiments of the present application provide an information processing method, device and equipment to solve the technical problem of large information acquisition time cost and inaccuracy in the prior art.

[0005] In a first aspect, an information processing method is provided in embodiments of the present application, comprising:

[0006] obtaining a shelf picture collected by a mobile robot in a target place; the target place is deployed with at least one shelf, the at least one shelf respectively displays at least one goods and is provided with a first information carrier at a respective display position of the at least one goods, the first information carrier carries goods information of the corresponding goods;

[0007] identifying the first information carrier in the shelf picture and the goods information carried by the first information carrier;

[0008] generating shelf display information based on the goods information of at least one goods belonging to the same shelf.

[0009] In a second aspect, an information processing method is provided in embodiments of the present application, comprising:

[0010] obtaining shelf distribution information based on a shelf distribution request for a target place;

[0011] outputting the shelf distribution information;

[0012] obtaining shelf display information of a target shelf based on a shelf display request for the target shelf in the shelf distribution information;

[0013] output the shelf display information; wherein the shelf display information is generated by identifying the first information carrier in the shelf picture and the article information carried by the first information carrier, and based on the article information corresponding to the same shelf; and the shelf distribution information is generated by identifying the second information carrier in the shelf picture and the shelf information carried by the second information carrier, and based on the respective shelf information of the at least one shelf.

[0014] In a third aspect, an information processing method is provided in the embodiments of the present application, and is applied to a mobile robot. The method comprises:

[0015] receiving a mapping instruction;

[0016] mapping in a target site, and constructing an electronic map of the target site;

[0017] receiving a patrol instruction, and determining a patrol area in the electronic map;

[0018] moving in the target site based on the patrol area, and collecting pictures at different collection points;

[0019] sending a collection result to a server, so that the server obtains a shelf picture of the target site, identifies a first information carrier in the shelf picture and article information carried by the first information carrier, and generates shelf display information based on article information of at least one article belonging to the same shelf.

[0020] In a fourth aspect, an information processing device is provided in the embodiments of the present application, and comprises:

[0021] a picture acquisition module, configured to acquire a shelf picture collected by a mobile robot in a target site; the target site is provided with at least one shelf, and the at least one shelf respectively displays at least one article and is provided with a first information carrier at a respective display position of the at least one article, and the first information carrier carries article information of a corresponding article;

[0022] an information identification module, configured to identify the first information carrier in the shelf picture and the article information carried by the first information carrier;

[0023] an information generation module, configured to generate shelf display information based on article information of at least one article belonging to the same shelf.

[0024] In a fifth aspect, an information processing device is provided in the embodiments of the present application, and comprises:

[0025] a first acquisition module, configured to acquire shelf distribution information based on a shelf distribution request for a target site;

[0026] The first output module is configured to output the shelf distribution information.

[0027] The second information acquisition module is configured to acquire, based on a shelf display request for a target shelf in the shelf distribution information, shelf display information of the target shelf.

[0028] The second output module is configured to output the shelf display information; wherein the shelf display information is generated by identifying a first information carrier in a shelf picture and article information carried by the first information carrier, and based on article information corresponding to a same shelf; and the shelf distribution information is generated by identifying a second information carrier in the shelf picture and shelf information carried by the second information carrier, and based on respective shelf information of the at least one shelf.

[0029] In a sixth aspect, an information processing device is provided in the embodiments of the present application, and includes:

[0030] The map construction module is configured to receive a mapping instruction, and to perform mapping in a target site to construct an electronic map of the target site.

[0031] The region determination module is configured to receive a patrol instruction, and to determine a patrol region in the electronic map.

[0032] The picture acquisition module is configured to move in the target site based on the patrol region, and to acquire pictures at different acquisition points.

[0033] The picture sending module is configured to send an acquisition result to a server, so that the server obtains a shelf picture of the target site, and identifies a first information carrier in the shelf picture and article information carried by the first information carrier, and generates shelf display information based on article information of at least one article belonging to a same shelf.

[0034] In a seventh aspect, an information processing system is provided in the embodiments of the present application, and includes a mobile robot and a server.

[0035] The mobile robot is configured to acquire pictures in a target site, and to send an acquisition result to the server; wherein the target site is deployed with at least one shelf, the at least one shelf respectively displays at least one article and is provided with a first information carrier at a respective display position of the at least one article, and the first information carrier carries article information of a corresponding article.

[0036] The server is configured to acquire a shelf picture acquired by the mobile robot, and to identify a first information carrier in the shelf picture and article information carried by the first information carrier, and to generate shelf display information based on article information of at least one article belonging to a same shelf.

[0037] In an eighth aspect, an embodiment of the present application provides a computing device, comprising a processing component and a storage component;

[0038] The storage component stores one or more computer instructions; the one or more computer instructions are used to be called and executed by the processing component, and realize the information processing method in the first aspect.

[0039] In a ninth aspect, an embodiment of the present application provides an electronic device, comprising a processing component, a display component and a storage component;

[0040] The storage component stores one or more computer instructions; the one or more computer instructions are used to be called and executed by the processing component, and realize the information processing method in the second aspect.

[0041] In a tenth aspect, an embodiment of the present application provides a mobile robot, comprising a processing component, an acquisition component and a storage component;

[0042] The storage component stores one or more computer instructions; the one or more computer instructions are used to be called and executed by the processing component, and realize the information processing method in the third aspect.

[0043] In an eleventh aspect, an embodiment of the present application provides a computer storage medium, which stores a computer program; the computer program is executed by a computer to realize the information processing method in the first aspect.

[0044] In a twelfth aspect, an embodiment of the present application provides a computer storage medium, which stores a computer program; the computer program is executed by a computer to realize the information processing method in the second aspect.

[0045] In a thirteenth aspect, an embodiment of the present application provides a computer storage medium, which stores a computer program; the computer program is executed by a computer to realize the information processing method in the third aspect.

[0046] In the embodiment of the present application, the target site has goods displayed in the shelves, and a first information carrier carrying the information of the goods is arranged at the display position of the goods. The mobile robot collects pictures in the target site and uploads them to the server. The server can obtain the shelf pictures in the target site. By identifying the shelf pictures, the first information carrier and the information of the goods carried by the first information carrier in the shelf pictures can be identified. Therefore, based on the information of at least one goods belonging to the same shelf, the shelf display information of the shelf can be automatically generated. Only the first information carrier is arranged at the display position of the goods in the shelf, and the information of the goods carried by the first information carrier can be automatically identified and obtained from the shelf pictures collected by the mobile robot, and the shelf display information can be automatically generated, thereby reducing the labor participation workload, and efficient and accurate information acquisition is realized.

[0047] These aspects or other aspects of the present application will be more apparent in the following description of the embodiments. BRIEF DESCRIPTION OF DRAWINGS

[0048] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or the prior art description. Obviously, the drawings in the following description are some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0049] Figure 1 A flow chart of one embodiment of an information processing method provided by the present application is shown;

[0050] Figure 2 A flow chart of another embodiment of an information processing method provided by the present application is shown;

[0051] Figure 3 A flow chart of another embodiment of an information processing method provided by the present application is shown;

[0052] Figure 4a A system interaction schematic diagram of the information processing process of the embodiment of the present application in an actual application is shown;

[0053] Figure 4b A flow interaction schematic diagram of the information processing process of the embodiment of the present application in an actual application is shown;

[0054] Figure 4c A schematic diagram of one implementation form of the shelf distribution information of the embodiment of the present application in an actual application is shown;

[0055] Figure 4d A schematic diagram of one implementation form of the shelf display information of the embodiment of the present application in an actual application is shown;

[0056] Figure 5 Fig. 1 shows a structural schematic diagram of one embodiment of an information processing apparatus provided by the present application;

[0057] Figure 6 Fig. 2 shows a structural schematic diagram of one embodiment of a computing device provided by the present application;

[0058] Figure 7 Fig. 3 shows a structural schematic diagram of another embodiment of an information processing apparatus provided by the present application;

[0059] Figure 8 Fig. 4 shows a structural schematic diagram of one embodiment of an electronic device provided by the present application;

[0060] Figure 9 Fig. 5 shows a structural schematic diagram of another embodiment of an information processing apparatus provided by the present application;

[0061] Figure 10 Fig. 6 shows a structural schematic diagram of one embodiment of a mobile robot provided by the present application. DETAILED DESCRIPTION

[0062] In order to make the personnel in the technical field better understand the scheme of the present application, the technical scheme in the embodiments of the present application will be clearly and completely described below in combination with the drawings in the embodiments of the present application.

[0063] In some of the processes described in the specification and claims of the present application and the above drawings, a plurality of operations appear in a specific order, but it should be clearly understood that these operations can be executed or performed in parallel or in a sequence different from that in which they appear in the present text, and the serial numbers of the operations, such as 101, 102, etc., are only used to distinguish different operations, and the serial numbers themselves do not represent any execution sequence. In addition, these processes can include more or fewer operations, and the operations can be executed or performed in sequence or in parallel. It should be noted that the descriptions of “first”, “second” and the like in the present text are used to distinguish different messages, devices, modules, etc., and do not represent the sequence, nor do “first” and “second” represent different types.

[0064] The technical scheme of the present application can be applied to various offline places where shelves are deployed, such as small convenience stores, large supermarkets, warehouses, and some warehouse-style shopping malls, etc.

[0065] With the advent of the intelligent era, the requirements of these offline places for digital operation are also higher and higher, and the digitalization of the shelves is an important part of it. The digitalization of the shelves at least needs to understand the display situation of the shelves, which is very important for the management of goods and the operation of the place. To understand the display situation of the shelves, it is necessary to know the commodity information of each commodity on the shelf, and these commodity information usually changes frequently. If the manual method is used to record the shelf display information, it will consume a lot of time cost, and may not be accurate due to the delay of updating.

[0066] In order to realize automatic, efficient and accurate acquisition of shelf display information, the inventors have proposed the technical scheme of the present application after a series of researches. In the embodiment of the present application, the goods are displayed in the shelves of the target place, and the first information carrier carrying the goods information is arranged at the display position of the goods. The mobile robot collects pictures in the target place and uploads them to the server. The server can obtain the shelf pictures in the target place. By identifying the shelf pictures, the first information carrier and the goods information carried by the first information carrier in the shelf pictures can be identified, so that the goods information of the goods belonging to the same shelf can be automatically generated. The goods display information of each shelf can be automatically generated. The technical scheme of the present application only needs to set the first information carrier at the display position of the goods in the shelf. The goods information carried by the first information carrier can be automatically identified and obtained through the shelf pictures collected by the mobile robot, which reduces the labor participation workload, so as to realize the efficient and accurate acquisition of the display information.

[0067] The technical scheme in the embodiments of the present application will be described clearly and completely in combination with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0068] Figure 1 The flowchart of one embodiment of the information processing method provided in the embodiments of the present application can be executed by the server. The method can include the following steps:

[0069] 101: Obtain the shelf pictures collected by the mobile robot in the target place.

[0070] The target site can be deployed with at least one shelf; each shelf can respectively display at least one item and be provided with a corresponding first information carrier at the respective display position of each item; the first information carrier carries the item information of the corresponding item. In actual application, the first information carrier can be a graphic code such as a bar code, a two-dimensional code or a three-dimensional code. Since a price tag corresponding to the item is usually deployed in the shelf, the first information carrier can be arranged in the price tag; the price tag can be an electronic price tag or a paper price tag, and the present application does not make specific limitation thereon. Of course, for the consideration of deployment cost, a paper price tag can be used.

[0071] The mobile robot is a machine device that can automatically perform work. It can accept human command, run a pre-programmed program, or act according to principles and guidelines formulated by artificial intelligence technology. It is usually a comprehensive system integrating environment perception, dynamic decision and planning, behavior control and execution, etc. The mobile robot can move autonomously, and the moving mode can be, for example, wheel type, track type, etc. The same as the prior art, the present application does not make limitation thereon. The mobile robot can accept corresponding instructions and perform corresponding operations.

[0072] In the embodiments of the present application, the mobile robot can move in the target site and collect pictures based on the inspection instruction, and upload the collection result to the server, so that the server can obtain the shelf pictures. The inspection instruction can be sent by the user or sent by the server to the mobile robot based on the user's request, etc.

[0073] The shelf pictures can include at least one picture. In actual application, a plurality of shelves in the target site can be arranged side by side, so the shelf pictures can refer to pictures including one or more complete shelves.

[0074] Optionally, the collection result of the mobile robot can include a plurality of pictures to be processed, and the server can perform repeated area detection and / or image splicing, cropping and other picture processing operations on the plurality of pictures to be processed to obtain the shelf pictures. These picture processing operations are conventional image processing operations, the same as the prior art, and will not be repeated here.

[0075] The mobile robot can automatically plan a path and automatically avoid obstacles in the target site during picture collection, and the specific control operation implementation is the same as the prior art, and the present application does not make specific limitation thereon.

[0076] In addition, as another optional mode, the shelves can also collect pictures of themselves, and the pictures to be processed in the collection result of the mobile robot are assumed to be first pictures, wherein the first pictures can include multiple pictures, and optionally, second pictures collected by the at least one shelf on the shelf itself can also be obtained; thus, the shelf pictures can be obtained based on the first pictures and the second pictures.

[0077] The first pictures and the second pictures can be subjected to repeated area detection and / or image splicing, clipping processing and other picture processing operations to obtain the shelf pictures, so that the obtained shelf pictures are more accurate. These picture processing operations are conventional image processing operations, which are the same as the prior art, and will not be described here.

[0078] In addition, as another optional mode, some image collection devices such as monitoring cameras can also be fixedly deployed at specific positions in the target site, for example, for pedestrian monitoring, and thus the fixedly deployed image collection devices can also be used to implement picture collection. Therefore, optionally, obtaining the shelf pictures collected by the mobile robot in the target site can include:

[0079] Obtaining first pictures collected by the mobile robot in the target site;

[0080] Obtaining third pictures collected by the image collection devices fixedly deployed in the target site;

[0081] Obtaining the shelf pictures based on the first pictures and the third pictures.

[0082] The pictures to be processed in the collection result of the mobile robot are assumed to be first pictures, and the third pictures collected by the image collection devices are assumed to be third pictures, wherein the image collection devices can include multiple image collection devices, and the third pictures can include multiple third pictures. The present application does not make specific limitations on this. Based on the first pictures and the third pictures, the shelf pictures can be obtained through repeated area detection and / or image splicing, clipping processing and other picture processing operations, so that the obtained shelf pictures are more accurate. These picture processing operations are conventional image processing operations, which are the same as the prior art, and will not be described here.

[0083] Of course, as another optional mode, the second pictures collected by the shelves or the third pictures collected by the image collection devices can be sent to the mobile robot. The mobile robot obtains the shelf pictures through picture processing operations based on the first pictures collected by itself, and then sends the shelf pictures to the server. The server can directly perform subsequent processing operations on the shelf pictures uploaded by the mobile robot.

[0084] In addition, as another optional mode, the server can also obtain the shelf picture based on the first picture collected by the mobile robot, the second picture collected by the shelf, and the third picture collected by the image collection device.

[0085] Of course, the shelf picture can also be obtained by the mobile robot based on the first picture collected by the mobile robot, the second picture collected by the shelf, and the third picture collected by the image collection device, and then uploaded to the server after processing.

[0086] 102: Identify the first information carrier in the shelf picture and the item information carried by the first information carrier.

[0087] Since the first information carrier is deployed at the display position of the item on each shelf, by identifying the shelf picture, the first information carrier in the shelf picture can be identified, and then the first information carrier is identified, that is, the item information of the item represented by each first information carrier can be obtained.

[0088] The item information may, for example, include item identification, and in addition, may also include item-related description information, such as item-related description information including item price, item origin, item attributes such as size, weight, or quantity, etc. when the item is sold as a commodity in a supermarket.

[0089] 103: Generate shelf display information based on the item information of at least one item belonging to the same shelf.

[0090] For each shelf, based on the item information of at least one item belonging to the shelf, the shelf display information corresponding to each shelf can be generated, and the item information of at least one item displayed in each shelf can be included in the shelf display information of each shelf. Wherein, at least one item belonging to the same shelf can refer to all items displayed in the shelf, that is, the shelf display information is generated based on the item information carried by all first information carriers belonging to the shelf identified.

[0091] As an optional mode, the item information may also include the display position of the item in the shelf, so the shelf display information can be generated based on the display position of at least one item belonging to the same shelf and the item-related description information including the item identification. Thus, through the shelf display information, the display position of the item in the shelf and the item identification and other item-related description information can be obtained.

[0092] As another alternative, the first carrier position of the first information carrier in the shelf picture can also be identified; since each first information carrier represents one item, the display position of the corresponding item in the shelf can be determined based on the first carrier position of the first information carrier. Therefore, in this alternative, the method can further comprise:

[0093] identifying the first carrier position of the first information carrier in the shelf picture;

[0094] determining the display position of the corresponding item in the shelf based on the first carrier position of the first information carrier;

[0095] Step 103 can be specifically generating the shelf display information based on the display position and the item information of at least one item belonging to the same shelf.

[0096] The shelf display information can be stored as structured information, of course, other ways such as graphical form can also be used, which will be described in detail in the following embodiments.

[0097] When the target site includes multiple shelves, in order to determine the items belonging to the same shelf, as an alternative, the item information can also include the shelf identifier of the shelf where the item is located, so that the items or first information carriers belonging to the same shelf can be determined based on the shelf identifier.

[0098] Since the display position of the item in the target site can change at any time, when the first information carrier is provided on the price tag, it can only include the item identifier and the item-related description, so that when the display position of the item changes, the corresponding price tag can be replaced and deployed in the new position, and the first information carrier does not need to be regenerated. Then, as another alternative, in order to determine the items belonging to the same shelf, a second information carrier carrying the shelf information of each shelf can be provided in each shelf; wherein the shelf information can at least include the shelf identifier, and in addition, the shelf attribute, for example, the category of the items displayed on the shelf, such as clothes, general merchandise, or hygiene and care, etc. In this alternative, the method can further comprise:

[0099] identifying the second carrier position of the second information carrier in the shelf picture;

[0100] determining the shelf display picture corresponding to each shelf in the shelf picture based on the second carrier position of different second information carriers;

[0101] Generating the shelf display information based on the item information of at least one item belonging to the same shelf can include:

[0102] generating the shelf display information based on the item information carried by the at least one first information carrier in the same shelf display picture.

[0103] wherein each first information carrier represents an item, and each shelf display picture can refer to a picture including only one complete shelf. Optionally, a first carrier position of each first information carrier in the shelf display picture where it is located can also be identified, so that the display position of the item in the shelf can be determined based on the first carrier position of the first information carrier. Thus, the shelf display information can be generated based on the item information carried by the at least one first information carrier in the same shelf display picture and the display position of the respective item.

[0104] Optionally, to facilitate the determination of the shelf display picture of each shelf, the deployment positions of the second information carriers in different shelves can be the same, for example, the second information carriers can be deployed at the left boundary of the shelf, and the picture area between two second information carriers can be attributed to the shelf identified by the left second information carrier. In practical applications, the second information carrier can be a graphic code such as a bar code, a two-dimensional code or a three-dimensional code.

[0105] In practical applications, the shelf usually includes multiple layers of supports, and the display position of the item can include the layer position of the item in the shelf and the arrangement position of the item in each layer. Optionally, the first carrier positions of different first information carriers can be clustered to determine the first information carriers located on the same layer of supports, so as to obtain the layer position of the first information carrier; then, the first carrier positions of the first information carriers located on the same layer can be sorted, so as to obtain the arrangement position of the first information carrier in the same layer, and the layer position and the arrangement position of the first information carrier can be taken as the layer position and the arrangement position of the item corresponding to the first information carrier in the shelf.

[0106] In the embodiment, the items are displayed in the shelves of the target site, and the first information carriers carrying the item information are deployed at the display positions of the items. The mobile robot collects the pictures in the target site and uploads them to the server, and the server can obtain the shelf pictures in the target site. By identifying the shelf pictures, the first information carriers in the shelf pictures and the item information carried by the first information carriers can be identified, so that the shelf display information of each shelf can be automatically generated based on the item information belonging to the same shelf. The technical solution of the present application only needs to set the first information carriers at the display positions of the items in the shelf, and the item information carried by the first information carriers can be automatically identified and obtained from the shelf pictures collected by the mobile robot, thereby reducing the workload of manual participation and achieving efficient and accurate acquisition of the display information.

[0107] Since multiple shelves can be deployed in the target site, in the case where multiple shelves are deployed in the target site, in order to facilitate understanding of the distribution information of the shelves in the target site, as described above, the second information carrier carrying the shelf information can be arranged in each shelf. Therefore, in some embodiments, the method can further include:

[0108] identifying the second information carrier in the shelf picture and the shelf information carried by the second information carrier;

[0109] generating the shelf distribution information based on the respective shelf information of the at least one shelf in the target site.

[0110] In the shelf information, the shelf identifier and the shelf attribute information can be included.

[0111] As an optional way, the shelf identifier can refer to the shelf position of the shelf in the target site, so that the shelf distribution information can be generated based on the shelf position and the shelf attribute of the at least one shelf.

[0112] Of course, as another optional way, the mobile robot performs picture collection at different collection points, and the collection coordinates of the different collection points in the target site can be recorded. Therefore, the method can further include:

[0113] determining the respective shelf positions of the at least one shelf in the target site based on the respective collection coordinates of the shelf pictures in the target site;

[0114] Then, generating the shelf distribution information based on the respective shelf information of the at least one shelf can include:

[0115] generating the shelf distribution information based on the respective shelf positions and the shelf information of the at least one shelf.

[0116] Optionally, the collection coordinates of the shelf picture can be used as the shelf position of the shelf in the target site.

[0117] When the shelf picture is obtained by splicing and / or clipping processing of at least one to-be-processed picture collected by the mobile robot, the collection coordinates of the shelf picture can be obtained by fusing the collection coordinates of the at least one to-be-processed picture corresponding thereto, such as using the average value of the collection coordinates of the at least one to-be-processed picture as the collection coordinates of the shelf picture.

[0118] In some embodiments, the method can further include:

[0119] identifying the goods in the shelf picture to obtain a goods cutout;

[0120] establishing a binding relationship between the goods cutout and the goods information;

[0121] The generating the shelf display information based on the item information of the at least one item belonging to the same shelf and the item cutout bound by the item information comprises:

[0122] The generating the shelf display information based on the item information of the at least one item belonging to the same shelf and the item cutout bound by the item information comprises:

[0123] In the embodiments of the present application, the items in the shelf picture can also be detected and recognized to obtain the item cutout. The item recognition manner is the same as the prior art, for example, a pre-trained model can be used to locate the item detection box in which the item in the shelf picture is located, and the picture area in the item detection box is taken as the item cutout.

[0124] The binding relationship between the item cutout and the item information can be established in the following manner:

[0125] The binding relationship between the item cutout and the item information is established based on the cutout position of the item cutout and the first carrier position of the first information carrier.

[0126] The binding relationship between the item cutout and the item information is established based on the cutout position of the item cutout and the first carrier position of the first information carrier.

[0127] The item cutout and the first information carrier can be considered as corresponding to the same item if the positions of the cutout position and the first carrier position are the same or the distance between the positions is within an error range, so that the binding relationship can be established. Furthermore, the shelf display information can be generated based on the item information of the at least one item belonging to the same shelf and the item cutout bound by the item information. The shelf display information can not only understand the item information, but also find the corresponding item cutout.

[0128] In addition, the cutout position of the item cutout and the first carrier position of the first information carrier can be obtained, and the cutout position and the first carrier position can be bound, so that the display position of the item can also be determined based on the cutout position. Therefore, in some embodiments, the method can further comprise:

[0129] The display position of the corresponding item in the shelf is determined based on the cutout position of the item cutout or the first carrier position of the first information carrier.

[0130] The generating the shelf display information based on the item information of the at least one item belonging to the same shelf and the item cutout bound by the item information comprises:

[0131] The shelf display information is generated based on the display positions, the item information and the item cutout of the at least one item belonging to the same shelf, respectively.

[0132] In some embodiments, the method can further comprise:

[0133] performing picture processing on the shelf picture based on the second carrier position of the different second information carrier to obtain a shelf display picture corresponding to the different shelf;

[0134] The generating of the shelf distribution information based on the respective shelf information of the at least one shelf comprises:

[0135] The generating of the shelf distribution information based on the respective shelf information of the at least one shelf and the corresponding shelf display picture.

[0136] That is, the shelf distribution information can include the shelf information of each shelf and the corresponding shelf display picture. Through the shelf distribution information, not only the shelf distribution situation in the target site can be understood, but also the shelf display picture of each shelf can be viewed.

[0137] The shelf display information and the shelf distribution information can be stored in a structured information form, and in addition, can be stored in a form of a display map or a distribution map.

[0138] Therefore, in some embodiments, the generating of the shelf distribution information based on the respective shelf information of the at least one shelf can comprise:

[0139] The shelf region segmentation in the electronic map of the target site based on the respective shelf information of the at least one shelf to obtain a shelf distribution map.

[0140] The electronic map of the target site can be a two-dimensional plane map of the target site, which can be constructed in advance, and of course can be constructed by a mobile robot, which will be introduced below.

[0141] In some embodiments, the generating of the shelf display information based on the item information of the at least one item belonging to the same shelf can comprise:

[0142] The generating of the shelf display map based on the item information of the at least one item belonging to the same shelf.

[0143] Optionally, the generating of the shelf display map can be specifically based on the display position and the item information of the at least one item belonging to the same shelf.

[0144] Optionally, the generating of the shelf display map can be specifically based on the display position, the item information and the item cutout of the at least one item belonging to the same shelf.

[0145] In addition, the method can further comprise:

[0146] Establishing an association between the shelf display map and the shelf distribution map to generate an item distribution map of the target site.

[0147] Due to the area distribution of different shelves shown in the shelf distribution map, each shelf in the shelf distribution map can be associated with a corresponding shelf arrangement map, so that for any shelf in the shelf distribution map, the associated shelf arrangement map is requested.

[0148] The article distribution map can have various use scenarios, for example, it can be provided to relevant personnel, such as the management personnel of the target site, or the customers entering the target site, and the like.

[0149] Based on the article distribution map, article searching and the like can be performed, and therefore, after obtaining the article distribution map, in an actual application, the method can further include:

[0150] outputting a shelf distribution map in the article distribution map;

[0151] receiving a search request for a target article;

[0152] finding a target shelf where the target article is located;

[0153] outputting a shelf arrangement map corresponding to the target shelf.

[0154] Among them, when a map output request for the article distribution map is received, the shelf distribution map in the article distribution map can be outputted; since the article distribution map is composed of the shelf distribution map and the shelf arrangement map, in response to the map output request, specifically, the shelf distribution map is outputted;

[0155] The search request can include an article identifier of the target article, since the article information includes the article identifier, the target shelf where the target article is located can be determined based on the article identifier, and then the shelf arrangement map corresponding to the target shelf can be found, and then the shelf arrangement map is outputted to facilitate user viewing.

[0156] In addition, the target article can also be marked in the shelf arrangement map, for example, a specific marker symbol is set at the arrangement position of the target article.

[0157] In yet another actual application, after obtaining the article distribution map, the method can further include:

[0158] outputting a shelf distribution map in the article distribution map;

[0159] receiving a navigation request of a user for a target article;

[0160] finding a target shelf where the target article is located;

[0161] generating a navigation route based on the shelf position of the target shelf and the user position;

[0162] outputting the navigation route in the shelf distribution map.

[0163] It should be noted that there can be various application scenarios based on the item distribution map, which are not specifically limited in the present application.

[0164] In addition, in some embodiments, the method can further include:

[0165] establishing an association relationship between the shelf display information and the shelf distribution information to obtain the item distribution information.

[0166] That is, the shelf display information and the shelf distribution information can be associated to obtain the item distribution information. The item distribution information includes the shelf distribution information and the shelf display information associated with each shelf in the shelf distribution information.

[0167] Since the second information carrier can be arranged in the shelf, the shelf identifier can be identified and obtained therefrom, so that the shelf display information corresponding to each shelf can be determined based on the shelf identifier, and then the association relationship with the shelf distribution information is established.

[0168] There can be various application scenarios for the item distribution information, for example, in some embodiments, the method can further include:

[0169] providing the item distribution information to relevant personnel.

[0170] These relevant personnel may, for example, refer to the management personnel of the target site, or the customers entering the target site, or the item provider, etc.

[0171] In addition, in some embodiments, the method can further include:

[0172] determining whether there is a display abnormality based on the item distribution information.

[0173] It can be determined whether there is a display abnormality in each shelf based on the shelf display information in the item distribution information.

[0174] Since the shelf display information can include the item information, the item cutout, and the display position of each item, the display abnormality may, for example, be that the display position of the item is different from the preset position; the item description information of the item is different from the preconfigured information, such as the price of the item being different from the preconfigured price; or the item cutout does not match the actual picture of the item, etc., to achieve detection of abnormal situations such as out-of-stock and misplaced goods, etc.

[0175] In addition, when there is a display abnormality, a display warning can be given to relevant personnel, and the warning method may, for example, be to send warning information to relevant personnel through various communication forms, etc., which are not specifically limited in the present application.

[0176] As can be known from the foregoing description, the shelf picture can be obtained by the mobile robot moving in the target site. In order to obtain a more accurate shelf picture, in some embodiments, the method can further include:

[0177] controlling the mobile robot to map in the target site and construct an electronic map of the target site;

[0178] determining a patrol area in the electronic map;

[0179] controlling the mobile robot to move in the target site based on the patrol area and collect pictures at different collection points.

[0180] Thus, the mobile robot can upload the collection result to the server, and the server can obtain the shelf picture therefrom.

[0181] The mapping and the construction of the electronic map by the mobile robot can be the same as the prior art, and thus will not be described herein.

[0182] Optionally, a mapping instruction can be sent to the mobile robot based on a user-triggered mapping request, so as to trigger the mobile robot to map in the target site and construct an electronic map of the target site in response to the mapping instruction.

[0183] The patrol area can be selected and determined in the electronic map by the user in combination with the shelf position. Therefore, optionally, the determination of the patrol area in the electronic map can be:

[0184] determining the patrol area in the electronic map based on a user selection operation.

[0185] After the electronic map is constructed, the electronic map can be displayed for the user to perform the selection operation.

[0186] After the patrol area is determined, the mobile robot can be controlled to move in the target site based on the patrol area to collect pictures at different collection points. The mobile robot can automatically perform path planning and obstacle avoidance processing and other control operations according to the patrol area.

[0187] Optionally, a patrol instruction can be sent to the mobile robot in response to a user-triggered patrol request, so as to control the mobile robot to move in the target site based on the patrol area and collect pictures at different collection points in response to the patrol instruction.

[0188] The patrol instruction can include the patrol area, and in addition, can include position information of the different collection points.

[0189] Of course, as other optional implementations, the training instructions can include an inspection step length, such as 1 meter, which can be determined by the mobile robot in combination with the inspection area and the inspection step length.

[0190] Since there can be aisles and other areas without shelves in the target site, the mobile robot will inevitably collect pictures that do not contain shelves during the picture collection process.

[0191] Therefore, as an optional way, obtaining the shelf pictures collected by the mobile robot in the target site can include:

[0192] Obtaining the to-be-processed pictures collected by the mobile robot in the target site;

[0193] Filtering invalid pictures based on the collection coordinates corresponding to the to-be-processed pictures to obtain the shelf pictures.

[0194] Since the mobile robot collects the to-be-processed pictures at different collection points, based on the constructed electronic map, the collection coordinates of different collection points can be determined, so if the collection coordinates are aisle coordinates, the to-be-processed picture can be considered as an invalid picture, and thus it can be filtered out from the collection result.

[0195] As another optional way, obtaining the shelf pictures collected by the mobile robot in the target site includes:

[0196] Obtaining the to-be-processed pictures collected by the mobile robot in the target site;

[0197] Performing information carrier recognition on the to-be-processed pictures;

[0198] Filtering invalid pictures based on the recognition result to obtain the shelf pictures.

[0199] Since the first information carrier is arranged in the shelf, in addition, a second information carrier can also be arranged, so by recognizing the to-be-processed pictures, if it is determined based on the recognition result that there is no any information carrier in the to-be-processed picture, the to-be-processed picture can be considered as an invalid picture, and thus it can be filtered out from the collection result.

[0200] Of course, the server can combine the collection coordinates and the recognition result to filter the invalid pictures.

[0201] In addition, the to-be-processed pictures remaining after filtering out the invalid pictures can be directly used as shelf pictures, and the shelf pictures can also be obtained by performing repeated area detection, image splicing and / or clipping processing and other picture processing operations on the to-be-processed pictures, which has been described in the foregoing and will not be repeated here.

[0202] Figure 2An information processing method provided by an embodiment of the present application is shown in a flowchart of another embodiment. The embodiment introduces the technical solution of the present application from the perspective of application of shelf display information and shelf distribution information. The method can include the following steps:

[0203] 201: Obtain shelf distribution information based on a shelf distribution request for a target site.

[0204] The technical solution of the embodiment can be executed by a user terminal. The shelf distribution request is triggered by a user. The shelf distribution information can be requested from a server based on the shelf distribution request.

[0205] The user terminal can be a user terminal corresponding to a staff member in the target site, a customer in the target site, or a provider of an item in the target site. Thus, the shelf distribution information can be provided to the staff member, the customer, or the provider of the item.

[0206] 202: Output the shelf distribution information.

[0207] 203: Obtain shelf display information of a target shelf based on a shelf display request for the target shelf in the shelf distribution information.

[0208] The shelf distribution information indicates the distribution of at least one shelf in the target site. The shelf distribution information can include shelf information such as shelf identification and shelf attributes, and information such as shelf display pictures.

[0209] Optionally, the shelf information in the shelf distribution information can be output first. In response to a picture viewing request for any shelf, the corresponding shelf display picture can be output again.

[0210] After the shelf distribution information is output, a trigger control for each shelf can also be output. The shelf display request can be generated by a trigger operation performed by a user on the trigger control of the target shelf. Based on the shelf display request, the shelf display information of the target shelf can be obtained from the server. The shelf identification can be included in the shelf display request. The server can search for the corresponding shelf display information based on the shelf identification.

[0211] 204: Output the shelf display information.

[0212] By outputting the shelf display information of the target shelf, the item display situation of the target shelf can be viewed. The shelf display information can include item information, display positions, and item cutouts.

[0213] Optionally, the shelf display information can be output first by outputting the item information and the display positions. In response to a picture viewing request for any item, the corresponding item cutout can be output again.

[0214] The shelf arrangement information is generated based on the item information corresponding to the same shelf by identifying the first information carrier in the shelf picture and the item information carried by the first information carrier.

[0215] The specific generation process of the shelf arrangement information and the shelf distribution information can be found in the foregoing embodiments, which will not be repeated here.

[0216] In some embodiments, the shelf distribution information can be a shelf distribution map generated based on the shelf information of at least one shelf and the electronic map of the target site.

[0217] The output shelf distribution information can include:

[0218] The output shelf distribution map and the shelf information corresponding to at least one shelf in the shelf distribution map.

[0219] Optionally, the shelf information can specifically include the shelf attributes such as category and the like.

[0220] The shelf arrangement information corresponding to the target shelf can be obtained based on the shelf arrangement request for the target shelf in the shelf distribution information.

[0221] The shelf arrangement information corresponding to the target shelf can be obtained by detecting the shelf arrangement request triggered by the target shelf in the shelf distribution map.

[0222] The shelf distribution map can show the distribution of different shelves and the category to which they belong. In the shelf distribution map, the user can perform a selection operation. The selection operation for the region where the target shelf is located in the shelf distribution map can trigger the shelf arrangement request. The target shelf can be any shelf.

[0223] Optionally, the shelf arrangement information can also be a shelf arrangement map generated based on at least the item information of at least one item belonging to the same shelf.

[0224] The output shelf arrangement information can specifically be a shelf arrangement map.

[0225] In actual applications, the user can understand the distribution of items in the target site, whether the items are out of stock, and the like by viewing the shelf distribution information and the shelf arrangement information. In combination with the traffic volume, the shelf in the region with large traffic volume can be determined, the shelf attributes and the shelf arrangement information of the shelf can be understood, and the popular or unpopular items can be determined. In summary, the shelf arrangement information and the shelf distribution information can be used for various applications, which are not limited in the present application.

[0226] Figure 3 An information processing method provided by an embodiment of the present application is shown in the flowchart of another embodiment. The embodiment describes the technical solution of the present application from the perspective of a mobile robot. The method can include the following steps:

[0227] 301: Receive a mapping instruction.

[0228] The mapping instruction can be triggered by a user directly or sent to the mobile robot by a server based on a user request.

[0229] 302: Perform mapping in a target site to construct an electronic map of the target site.

[0230] 303: Receive an inspection instruction and determine an inspection area in the electronic map.

[0231] The inspection instruction can include the inspection area, and in addition, can include a collection step length to facilitate the determination of a collection point by the mobile robot.

[0232] Therefore, the method can further include:

[0233] determining the collection step length based on the inspection instruction;

[0234] planning the collection point based on the collection step length.

[0235] 304: Move in the target site based on the inspection area and collect pictures at different collection points.

[0236] The mobile robot can also automatically plan a moving path based on the inspection area and perform obstacle avoidance processing and other operations.

[0237] 305: Send the collection result to the server to obtain a shelf picture of the target site, identify a first information carrier and item information carried by the first information carrier in the shelf picture, and generate shelf display information based on the item information of at least one item belonging to the same shelf.

[0238] In this embodiment, the mobile robot can quickly complete the map construction and picture collection of the target site, automatically implement picture collection in the target site, and be efficient and accurate. Therefore, the server can generate shelf display information in the target site based on the shelf picture, automatically implement the semanticization and visualization of the shelf display, reduce the workload of manual participation, and thus achieve efficient and accurate information acquisition.

[0239] In some embodiments, the picture collection at different collection points can include:

[0240] The pictures are collected at different collection points and collection coordinates are determined; the collection coordinates are used to filter invalid pictures in the collection result by the server to obtain the shelf pictures.

[0241] The collection coordinates can be sent to the server at the same time as the pictures to be processed in the collection result, so that the server performs picture filtering and the like.

[0242] In some embodiments, the method can further include:

[0243] The pictures collected by the image collection device are obtained; the image collection device is fixedly deployed in the target site;

[0244] The collection result is obtained based on the pictures collected at different collection points and the pictures collected by the image collection device.

[0245] Optionally, the collection result can directly include the pictures collected by the mobile robot at different collection points and the pictures collected by the image collection device.

[0246] In addition, as another optional way, the shelf pictures can be obtained based on the pictures collected at different collection points and the pictures collected by the image collection device;

[0247] The shelf pictures are sent to the server as the collection result.

[0248] That is, the mobile robot can perform picture processing operations such as repeated area detection and / or image stitching, cropping, and the like on the pictures collected at different collection points and the pictures collected by the image collection device to obtain the shelf pictures, and then upload them to the server; of course, the pictures collected at different collection points and the pictures collected by the image collection device can also be directly sent to the server, and the server can perform image processing operations to obtain the shelf pictures.

[0249] The pictures collected by the mobile robot at different collection points are assumed to be first pictures, which can include multiple pictures, and the pictures collected by the image collection device are assumed to be third pictures, which can include multiple pictures. The process of processing the first pictures and the third pictures to obtain the shelf pictures can be referred to the foregoing description, and will not be described here.

[0250] In addition, in some embodiments, the shelf can also collect pictures of itself, which are assumed to be second pictures. Therefore, the mobile robot can also obtain the second pictures collected by at least one shelf on the shelf itself; and the collection result can be obtained based on the first pictures collected at different collection points and the second pictures.

[0251] The collection result can be constituted by the first picture and the second picture, and of course can be constituted by a shelf picture obtained by performing picture processing based on the first picture and the second picture.

[0252] The technical solution of the embodiment of the application can be applied to offline places such as supermarkets, convenience stores, and warehouse-style shopping malls that provide commodity sales, and the goods are specifically commodities for sale. The technical solution of the embodiment of the application can automatically generate the shelf display information and the shelf distribution information in the offline place, without manual recording, reducing the amount of human participation, and achieving efficient and accurate information acquisition.

[0253] The application further provides an information processing system, which can include a mobile robot and a server;

[0254] The mobile robot is configured to collect pictures in a target place and send a collection result to the server. The target place is provided with at least one shelf, and the at least one shelf displays at least one item and is provided with a first information carrier at a display position of each item. The first information carrier carries item information of the corresponding item.

[0255] The server is configured to obtain a shelf picture collected by the mobile robot, identify the first information carrier in the shelf picture and the item information carried by the first information carrier, and generate shelf display information based on the item information of at least one item belonging to the same shelf.

[0256] The specific operations performed by the mobile robot can be found in Figure 3 The specific operations performed by the server can be found in Figure 1 The specific operations performed by the server can be found in

[0257] In addition, the system can further include a user terminal.

[0258] The server can further be configured to send the shelf display information to the user terminal.

[0259] The user terminal can be configured to output the shelf display information.

[0260] Of course, the user terminal can send a shelf distribution request for a target place to the server to obtain shelf distribution information and output the shelf distribution information. Based on a shelf display request for a target shelf in the shelf distribution information, the target shelf display information is obtained and output. The specific operations can be found in Figure 2 The specific operations performed by the server can be found in

[0261] For ease of understanding, as Figures 4a to 4dThe distribution shows a schematic diagram of an information processing process of the technical solution in an actual application.

[0262] Figure 4a In the embodiment, the mobile robot 41 can construct an electronic map in the target site based on the mapping instruction, and send the electronic map to the server 42. The server 42 can display the electronic map for a user to view or send the electronic map to the first user terminal 43. Since the first user terminal 43 displays the electronic map for the user to view, the user can select an inspection area therefrom. The server 42 can send an inspection instruction to the mobile robot 41 based on the inspection area. The mobile robot 41 can move in the target site based on the inspection instruction, and collect pictures at different collection points. The mobile robot 41 can upload the collection results to the server 42.

[0263] In the embodiment, the first information carrier and the second information carrier can be a graphic code such as a two-dimensional code, a bar code or a three-dimensional code. The first information carrier can be specifically an item code carrying item information, and the second information carrier can be specifically a shelf code carrying shelf information. The deployment positions of the shelf codes on each shelf can be the same, and the item codes are deployed at the display positions of the corresponding items.

[0264] Figure 4b The processing process of the server for the collection results is shown. The server can first filter out invalid pictures in the collection results through graphic code recognition and / or in combination with collection coordinates, to obtain shelf pictures 401. Then, the server detects and recognizes the shelf pictures, and based on the shelf information carried by the recognized shelf codes, can perform shelf segmentation to generate shelf distribution information 402. The shelf distribution information can be a shelf distribution map, which can be specifically as shown in Figure 4c In the embodiment, the processing process of the server for the collection results is shown. The server can first filter out invalid pictures in the collection results through graphic code recognition and / or in combination with collection coordinates, to obtain shelf pictures 401. Then, the server detects and recognizes the shelf pictures, and based on the shelf information carried by the recognized shelf codes, can perform shelf segmentation to generate shelf distribution information 402. The shelf distribution information can be a shelf distribution map, which can be specifically as shown in Figure 4d

[0265] As shown in Figure 4a ​As shown in the figure, after the server 42 obtains the shelf display information and the shelf distribution information, the server 42 can provide the first user terminal 43 or the second user terminal 44 for the corresponding personnel to view, etc.

[0266] Figure 5 An embodiment of an information processing device provided by the present application is shown in the structural schematic diagram. The device can include:

[0267] The picture acquisition module 501 is configured to acquire a shelf picture collected by a mobile robot in a target site. The target site is provided with at least one shelf, and the at least one shelf is arranged with at least one item and provided with a first information carrier at a respective display position of the at least one item. The first information carrier carries item information of the corresponding item.

[0268] The information identification module 502 is configured to identify the first information carrier in the shelf picture and the item information carried by the first information carrier.

[0269] The information generation module 503 is configured to generate shelf display information based on the item information of the at least one item belonging to the same shelf.

[0270] In some embodiments, the information identification module is further configured to identify a first carrier position of the first information carrier in the shelf picture, and determine a display position of the corresponding item in the shelf based on the first carrier position of the first information carrier.

[0271] The information generation module can be specifically configured to generate the shelf display information based on the display position and the item information of the at least one item belonging to the same shelf.

[0272] In some embodiments, the information identification module is further configured to identify the item in the shelf picture, obtain an item cutout, and establish a binding relationship between the item cutout and the item information.

[0273] The information generation module can be specifically configured to generate the shelf display information based on the item information of the at least one item belonging to the same shelf and the item cutout bound to the item information.

[0274] In some embodiments, the information identification module establishing the binding relationship between the item cutout and the item information can include: identifying a cutout position of the item cutout in the shelf picture and a first carrier position of the first information carrier in the shelf picture; and establishing the binding relationship between the item cutout and the item information based on the cutout position of the item cutout and the first carrier position of the first information carrier.

[0275] In some embodiments, the information generation module can be specifically used to determine the display position of the corresponding item on the shelf based on the cutout position of the item cutout or the first carrier position of the first information carrier; and to generate shelf display information based on the display position, item information and item cutout of at least one item belonging to the same shelf.

[0276] In some embodiments, each of the at least one shelf is provided with a second information carrier carrying information about its respective shelf;

[0277] The information recognition module is also used to identify the second information carrier in the shelf image and the shelf information carried by the second information carrier;

[0278] The information generation module is also used to generate shelf distribution information based on the shelf information of at least one shelf.

[0279] In some embodiments, the information generation module is also used to determine the location of at least one shelf in the target location based on the acquisition coordinates corresponding to the shelf image in the target location;

[0280] The information generation module generates shelf distribution information based on the shelf information of at least one shelf. This can include generating shelf distribution information based on the shelf location and shelf information of at least one shelf.

[0281] In some embodiments, the information recognition module is further used to identify the position of the second information carrier in the shelf image; and based on the position of the second carrier of different second information carriers, to determine the shelf display images corresponding to different shelves in the shelf image.

[0282] The information generation module generates shelf display information based on the item information of at least one item belonging to the same shelf. This can include generating shelf display information based on the item information carried by at least one first information carrier located in the same shelf display image.

[0283] In some embodiments, the device may further include:

[0284] The image processing module is used to process shelf images based on the location of the second carrier of different second information carriers to obtain shelf display images corresponding to different shelves.

[0285] The information generation module can generate shelf distribution information based on the shelf information of at least one shelf, which may include generating shelf distribution information based on the shelf information of at least one shelf and the corresponding shelf display images.

[0286] In some embodiments, the information generation module generates shelf distribution information based on the shelf information of at least one shelf, including:

[0287] based on the shelf information of the at least one shelf, performing shelf region segmentation in the electronic map of the target site to obtain a shelf distribution map;

[0288] The information generation module generates the shelf arrangement information based on the item information of the at least one item belonging to the same shelf can include: generating a shelf arrangement map based on the item information of the at least one item belonging to the same shelf.

[0289] In addition, the information generation module is further configured to establish an association between the shelf arrangement map and the shelf distribution map, and generate an item distribution map of the target site.

[0290] In some embodiments, the apparatus can further include:

[0291] The first processing module is configured to output the shelf distribution map in the item distribution map, receive a search request for a target item, and find a target shelf where the target item is located.

[0292] In some embodiments, the apparatus can further include:

[0293] The association module is configured to establish an association between the shelf arrangement information and the shelf distribution information, and obtain the item distribution information.

[0294] In some embodiments, the apparatus can further include:

[0295] The second processing module is configured to provide the item distribution information to a relevant person.

[0296] In some embodiments, the apparatus can further include:

[0297] Based on the item distribution information, it is determined whether there is an arrangement abnormality.

[0298] In some embodiments, the apparatus can further include:

[0299] The control module is configured to control the mobile robot to perform mapping in the target site to construct an electronic map of the target site, determine a patrol area in the electronic map, control the mobile robot to move in the target site based on the patrol area, and perform picture collection at different collection points.

[0300] In some embodiments, the information acquisition module can acquire the shelf pictures collected by the mobile robot in the target site, which can include: acquiring pictures to be processed collected by the mobile robot in the target site, and filtering invalid pictures based on the collection coordinates corresponding to the pictures to be processed to obtain the shelf pictures.

[0301] In some embodiments, the information acquisition module acquires the shelf picture collected by the mobile robot at the target site can include: acquiring a to-be-processed picture collected by the mobile robot at the target site; performing information carrier identification on the to-be-processed picture; filtering invalid pictures based on the identification result to obtain the shelf picture.

[0302] Figure 5 The information processing device can perform Figure 1 The information processing method of the embodiments described above has the same implementation principles and technical effects, which will not be described again. The specific manner in which each module or unit of the information processing device in the above embodiments performs operations has been described in detail in the embodiments related to the method, and will not be described in detail here.

[0303] In one possible design, Figure 5 The information processing device of the embodiments described above can be implemented as a computing device, such as a server, a personal computer, a mobile phone, a tablet computer, a network device, a terminal device, or the like. Figure 6 As shown in the embodiments, the computing device can include a storage component and a processing component.

[0304] The storage component 601 stores one or more computer instructions, wherein the one or more computer instructions are called and executed by the processing component 601 to implement the information processing method of the embodiments described above. Figure 1

[0305] Of course, the computing device can also include other components, such as an input / output interface, a communication component, and the like. The input / output interface provides an interface between the processing component and a peripheral interface module, which can be an output device, an input device, and the like. The communication component is configured to facilitate wired or wireless communication between the computing device and other devices.

[0306] The computing device can be a physical device or an elastic computing host provided by a cloud computing platform, and the computing device can be a cloud server. The processing component, the storage component, and the like can be basic server resources rented or purchased from the cloud computing platform.

[0307] The embodiments of the present application also provide a computer readable storage medium storing a computer program, and the computer program can implement the information processing method of the embodiments described above when executed by a computer. Figure 1

[0308] Figure 7 An embodiment of an information processing device provided by the embodiments of the present application is shown in a structural schematic diagram. The device can include:

[0309] The first acquisition module 701 is configured to acquire shelf distribution information based on a shelf distribution request for a target site.

[0310] ​​The first output module 702 is configured to output the shelf distribution information.

[0311] The second acquisition module 703 is configured to acquire shelf display information of a target shelf based on a shelf display request for the target shelf in the shelf distribution information.

[0312] The second output module 704 is configured to output the shelf display information, wherein the shelf display information is generated by identifying a first information carrier in a shelf picture and article information carried by the first information carrier, and based on article information corresponding to the same shelf.

[0313] In some embodiments, the shelf distribution information is a shelf distribution map generated based on the shelf information of the at least one shelf and an electronic map of the target site.

[0314] The first output module can specifically output the shelf distribution map, and display the shelf information corresponding to the at least one shelf in the shelf distribution map.

[0315] The second acquisition module can specifically detect a shelf display request triggered for a target shelf in the shelf distribution map, and acquire the shelf display information of the target shelf.

[0316] Figure 7 The information processing apparatus can execute Figure 2 The information processing method of the embodiments described above has the same implementation principles and technical effects, which will not be described in detail. The specific manner in which each module or unit of the information processing apparatus in the above embodiments performs operations has been described in detail in the embodiments of the method, and will not be described in detail here.

[0317] In one possible design, Figure 7 The information processing apparatus of the embodiments described above can be implemented as an electronic device, such as a computer, a mobile phone, a tablet computer, a personal digital assistant, a media player, a navigation device, a game console, a television, a camera, a wearable device, a vehicle, a home appliance, a medical device, a security device, a robot, a drone, a virtual reality device, an augmented reality device, a server, a cloud computing device, or any combination thereof. Figure 8 As shown in the figure, the electronic device can include a storage component 801, a display component 802, and a processing component 803.

[0318] The storage component 801 stores one or more computer instructions, wherein the one or more computer instructions are called and executed by the processing component 803 to implement the information processing method of the embodiments described above. Figure 2 The processing component 803 is configured to execute the one or more computer instructions stored in the storage component 801.

[0319] The display component 802 is responsible for outputting the shelf distribution information or the shelf display information based on the triggering of the processing component 803.

[0320] Of course, the electronic device can also include other components, such as an input / output interface, a communication component, etc. The input / output interface provides an interface between the processing component and peripheral interface modules, which can be output devices, input devices, etc. The communication component is configured to facilitate wired or wireless communication between the computing device and other devices, etc.

[0321] In actual applications, the electronic device can be, for example, a personal computer, a tablet computer, or a mobile terminal such as a mobile phone.

[0322] The embodiments of the present application also provide a computer readable storage medium storing a computer program, which can implement the information processing method of the above Figure 2 embodiments.

[0323] Figure 9 An information processing device provided by an embodiment of the present application is shown in a structural schematic diagram of another embodiment. The device can include:

[0324] The map construction module 901 is configured to receive a mapping instruction, perform mapping in a target site, and construct an electronic map of the target site.

[0325] The region determination module 902 is configured to receive a patrol instruction and determine a patrol region in the electronic map.

[0326] The picture acquisition module 903 is configured to move in the target site based on the patrol region and acquire pictures at different acquisition points.

[0327] The picture sending module 904 is configured to send the acquisition result to a server, so that the server obtains a shelf picture of the target site, identifies a first information carrier and article information carried by the first information carrier in the shelf picture, and generates shelf display information based on the article information of at least one article belonging to the same shelf.

[0328] In some embodiments, the device can also include:

[0329] The point planning module is configured to determine an acquisition step length based on the patrol instruction and plan acquisition points based on the acquisition step length.

[0330] In some embodiments, the picture acquisition module can acquire pictures at different acquisition points and determine acquisition coordinates, where the acquisition coordinates are used by the server to filter invalid pictures in the acquisition result to obtain the shelf picture.

[0331] In one possible design, Figure 9 The information processing device of the embodiments shown can be configured in a mobile robot, such as Figure 10As shown, the mobile robot can include a storage component 1001, a collection component 1002, and a processing component 1003.

[0332] The storage component 1001 stores one or more computer instructions, wherein the one or more computer instructions are called and executed by the processing component 1003 to implement the information processing method of the embodiments as described above. Figure 3 The information processing method of the embodiments as described above.

[0333] The collection component 1002 is responsible for picture collection based on the triggering of the processing component 1003.

[0334] Of course, the mobile robot can also include other components, such as an input / output interface, a communication component, etc. The input / output interface provides an interface between the processing component and peripheral interface modules, which can be output devices, input devices, etc. The communication component is configured to facilitate wired or wireless communication between the computing device and other devices, etc.

[0335] In addition, those skilled in the art can understand that the mobile robot can also include walking components, special-shaped shells, and other necessary components, which are not limited in the present application.

[0336] In addition, the embodiments of the present application also provide a computer readable storage medium storing a computer program, which can implement the information processing method of the embodiments as described above when executed by a computer. Figure 3 The information processing method of the embodiments as described above.

[0337] The processing component as involved in some embodiments above can include one or more processors to execute computer instructions to complete all or part of the steps in the above method. Of course, the processing component can also be one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors or other electronic elements for executing the above method.

[0338] The storage component can be configured to store various types of data to support operations in the device. The storage component can be implemented by any type of volatile or non-volatile storage device or their combination, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk.

[0339] The display component can be an electroluminescence (EL) element, a liquid crystal display or a micro display with similar structure, or a retina display or a similar laser scanning display.

[0340] The acquisition component can be a camera or a sensor that can realize image acquisition.

[0341] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described system, device and unit can refer to the corresponding processes in the foregoing method embodiments, which will not be described here.

[0342] The device embodiments described above are only schematic, and the units described as separate components can or can not be physically separated, and the components displayed as units can or can not be physical units, i.e., they can be located in one place or distributed on multiple network units. Part or all of the modules can be selected according to actual needs to achieve the purpose of the embodiment. Those skilled in the art can understand and implement without creative labor.

[0343] From the above description of the embodiments, those skilled in the art can clearly understand that the embodiments can be realized by means of software and necessary general hardware platforms, and of course can also be realized by hardware. Based on such understanding, the above technical solutions can be embodied in the form of software products, which can be stored in a computer readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and include a number of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute the methods described in each embodiment or some parts of the embodiments.

[0344] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for some technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. An information processing method characterized by comprising: The method comprises: obtaining a shelf picture collected by a mobile robot in a target site; the target site is provided with at least one shelf, and the at least one shelf is respectively provided with at least one article and is provided with a first information carrier at a respective display position of the at least one article, the first information carrier carrying article information of the corresponding article; the at least one shelf is respectively provided with a second information carrier carrying respective shelf information; the deployment positions of the second information carriers in different shelves are the same; identifying the first information carrier in the shelf picture and the article information carried by the first information carrier; generating shelf display information based on the article information of at least one article belonging to the same shelf; The method further comprises: identifying the second information carrier in the shelf picture and the second carrier position of the second information carrier in the shelf picture; determining the respective shelf display pictures of different shelves in the shelf picture based on the second carrier positions of different second information carriers; The method further comprises: generating shelf display information based on the article information carried by at least one first information carrier in the same shelf display picture; The method further comprises: identifying the article in the shelf picture to obtain an article cutout; identifying the cutout position of the article cutout in the shelf picture and the first carrier position of the first information carrier in the shelf picture; establishing a binding relationship between the article cutout and the article information based on the cutout position of the article cutout and the first carrier position of the first information carrier; determining the display position of the corresponding article in the shelf based on the cutout position of the article cutout or the first carrier position of the first information carrier; The method further comprises: generating shelf display information based on the display position, article information and article cutout of at least one article belonging to the same shelf.

2. The method of claim 1, wherein, The method further comprises: identifying the first carrier position of the first information carrier in the shelf picture; determining the display position of the corresponding article in the shelf based on the first carrier position of the first information carrier; The method further comprises: generating shelf display information based on the display position and article information of at least one article belonging to the same shelf.

3. The method of claim 1, wherein, The method further comprises: identifying the second information carrier in the shelf picture and the shelf information carried by the second information carrier; generating shelf distribution information based on the respective shelf information of the at least one shelf.

4. The method of claim 3, wherein, The method further comprises: determining the shelf position of the at least one shelf in the target site based on the collection coordinates of the shelf picture in the target site; The method further comprises: generating shelf distribution information based on the respective shelf position and shelf information of the at least one shelf.

5. The method of claim 3, wherein, The method further comprises: Based on the location of the second carrier of different second information carriers, image processing is performed on the shelf images to obtain shelf display images corresponding to different shelves; The process of generating shelf distribution information based on the shelf information of at least one shelf includes: Based on the shelf information and corresponding shelf display images of at least one shelf, generate shelf distribution information.

6. The method of claim 3, wherein, The process of generating shelf distribution information based on the shelf information of each of the at least one shelf includes: Based on the shelf information of each of the at least one shelf, the shelf area is segmented in the electronic map of the target location to obtain a shelf distribution map; The process of generating shelf display information based on item information belonging to at least one item in the same shelf includes: Generate a shelf display map based on item information belonging to at least one item in the same shelf; The method further includes: Establish the association between the shelf display map and the shelf distribution map to generate an item distribution map of the target location.

7. The method of claim 6, wherein, Also includes: Output the shelf distribution map in the item distribution map; Receive search requests for the target item; Locate the target shelf where the target item is located; Output the shelf display map corresponding to the target shelf.

8. The method of claim 3, wherein, Also includes: Establish the association between the shelf display information and the shelf distribution information to obtain the item distribution information.

9. The method of claim 8, wherein, Also includes: The distribution information of the items will be provided to the relevant personnel.

10. The method of claim 8, wherein, Also includes: Based on the item distribution information, determine whether there are any display anomalies.

11. The method of claim 1, wherein, Also includes: The mobile robot is controlled to conduct surveying and mapping at the target location, thereby constructing an electronic map of the target location. Determine the inspection area in the electronic map; The mobile robot is controlled to move within the target location based on the inspection area and to collect images at different collection points.

12. The method of claim 1, wherein, The acquisition of shelf images collected by the mobile robot at the target location includes: Acquire images to be processed from the target location collected by the mobile robot. Invalid images are filtered out based on the acquisition coordinates corresponding to the images to be processed to obtain shelf images.

13. The method of claim 1, wherein, The acquisition of shelf images collected by the mobile robot at the target location includes: Acquire images to be processed from the target location collected by the mobile robot. The information carrier of the image to be processed is identified; Invalid images are filtered based on the recognition results to obtain shelf images.

14. The method of claim 1, wherein, The acquisition of shelf images collected by the mobile robot at the target location includes: Acquire the first image captured by the mobile robot at the target location; Obtain the second image of each of the at least one shelf, which is captured by the shelf itself. Based on the first image and the second image, a shelf image is obtained.

15. The method of claim 1, wherein, The acquisition of shelf images collected by the mobile robot at the target location includes: Acquire the first image captured by the mobile robot at the target location; Acquire a third image obtained by an image acquisition device permanently deployed at the target location; Based on the first image and the third image, a shelf image is obtained.

16. An information processing method characterized by comprising: include: Based on the shelf distribution request for the target location, obtain shelf distribution information; Output the shelf distribution information; obtaining, based on a shelf display request for a target shelf in the shelf distribution information, shelf display information of the target shelf; outputting the shelf display information; wherein the shelf display information is generated by identifying a first information carrier in a shelf picture and item information carried by the first information carrier, and based on the item information in the same shelf display picture; the shelf display information is generated based on at least one item in the same shelf corresponding to a display position, item information and item cutout; the item cutout is obtained by identifying the item in the shelf picture; the cutout position of the item cutout in the shelf picture and the first carrier position of the first information carrier in the shelf picture are obtained based on the identification; the display position is determined based on the cutout position of the item cutout or the first carrier position of the first information carrier; the binding relationship between the item information and the item cutout is established based on the cutout position of the item cutout and the first carrier position of the first information carrier; the shelf display pictures corresponding to different shelves are determined by identifying the second carrier position of the second information carrier in the shelf picture and based on the second carrier positions of different second information carriers; the deployment positions of the second information carriers in different shelves are the same; the shelf distribution information is generated by identifying the second information carrier in the shelf picture and the shelf information carried by the second information carrier, and based on the shelf information of the at least one shelf.

17. The method of claim 16, wherein, The shelf distribution information is a shelf distribution map generated based on the shelf information of the at least one shelf and the electronic map of the target site; The outputting the shelf distribution information comprises: outputting the shelf distribution map and displaying the shelf information corresponding to the at least one shelf in the shelf distribution map; The obtaining, based on the shelf display request for the target shelf in the shelf distribution information, the shelf display information of the target shelf comprises: detecting a shelf display request triggered for a target shelf in the shelf distribution map, and obtaining the shelf display information of the target shelf.

18. An information processing method characterized by comprising: The method applied to a mobile robot comprises: receiving a mapping instruction; mapping in a target site to construct an electronic map of the target site; receiving a patrol instruction to determine a patrol area in the electronic map; moving in the target site based on the patrol area and collecting pictures at different collection points; The collection result is sent to the server to obtain a shelf picture of the target place, identify a first information carrier in the shelf picture and article information carried by the first information carrier, identify a second carrier position of a second information carrier in the shelf picture, determine shelf display pictures corresponding to different shelves in the shelf picture based on second carrier positions of different second information carriers, determine that deployment positions of the second information carriers in different shelves are the same, generate shelf display information based on article information carried by at least one first information carrier in the same shelf display picture, identify an article in the shelf picture, obtain an article cutout, identify a cutout position of the article cutout in the shelf picture and a first carrier position of the first information carrier in the shelf picture, establish a binding relationship between the article cutout and the article information based on the cutout position of the article cutout and the first carrier position of the first information carrier, determine a display position of a corresponding article in a shelf based on the cutout position of the article cutout or the first carrier position of the first information carrier, and generate shelf display information based on the display positions, the article information and the article cutout of at least one article belonging to the same shelf.

19. The method of claim 18, wherein, Also includes: Determine a collection step length based on the inspection instruction; Plan a collection point based on the collection step length.

20. The method of claim 18, wherein, The picture collection at different collection points includes: Collect pictures at different collection points and determine collection coordinates; wherein the collection coordinates are used by the server to filter invalid pictures in the collection result to obtain shelf pictures.

21. The method of claim 18, wherein, Also includes: Obtain pictures collected by an image collection device; wherein the image collection device is fixedly deployed in the target place; Obtain the collection result based on the pictures collected at different collection points and the pictures collected by the image collection device.

22. An information processing apparatus comprising: Includes: A picture acquisition module is configured to acquire shelf pictures collected by a mobile robot in a target place; The target place is deployed with at least one shelf, the at least one shelf respectively displays at least one article and is provided with a first information carrier at a respective display position of the at least one article, the first information carrier carries article information of a corresponding article, the at least one shelf is respectively provided with a second information carrier carrying respective shelf information, and the deployment positions of the second information carriers in different shelves are the same; An information identification module is configured to identify the first information carrier in the shelf picture and the article information carried by the first information carrier; An information generation module is configured to generate shelf display information based on the article information of at least one article belonging to the same shelf; The information identification module is further configured to identify a second carrier position of a second information carrier in the shelf picture, and determine shelf display pictures corresponding to different shelves in the shelf picture based on second carrier positions of different second information carriers; The information generation module is further configured to generate shelf display information based on article information carried by at least one first information carrier in the same shelf display picture; The information recognition module is further configured to recognize the goods in the shelf picture, and obtain a goods cutout; recognize a cutout position of the goods cutout in the shelf picture and a first carrier position of the first information carrier in the shelf picture; Based on the cutout position of the goods cutout and the first carrier position of the first information carrier, a binding relationship between the goods cutout and the goods information is established; Based on the cutout position of the goods cutout or the first carrier position of the first information carrier, a display position of the corresponding goods in the shelf is determined; The information generation module is further configured to generate shelf display information based on the display positions, the goods information and the goods cutout of at least one goods belonging to the same shelf.

23. An information processing apparatus, comprising: Comprise: The first acquisition module is configured to acquire shelf distribution information based on a shelf distribution request for a target site; The first output module is configured to output the shelf distribution information; The second information acquisition module is configured to acquire shelf display information of a target shelf based on a shelf display request for the target shelf in the shelf distribution information; The second output module is configured to output the shelf display information; wherein the shelf display information is generated by recognizing a first information carrier in a shelf picture and goods information carried by the first information carrier, and based on the goods information in the same shelf display picture; the shelf display information is generated based on the display positions, the goods information and the goods cutout of at least one goods belonging to the same shelf, the goods cutout is obtained based on the recognition of the goods in the shelf picture, the cutout position of the goods cutout in the shelf picture and the first carrier position of the first information carrier in the shelf picture are obtained based on the recognition, the display position is determined based on the cutout position of the goods cutout or the first carrier position of the first information carrier, the binding relationship between the goods information and the goods cutout is established based on the cutout position of the goods cutout and the first carrier position of the first information carrier; different shelf display pictures corresponding to different shelves are determined based on the second carrier positions of different second information carriers by recognizing the second carrier positions of the second information carriers in the shelf pictures; the deployment positions of the second information carriers in different shelves are the same; the shelf distribution information is generated based on the shelf information of the at least one shelf by recognizing the second information carriers in the shelf pictures and the shelf information carried by the second information carriers.

24. An information processing apparatus comprising: Comprise: The map construction module is configured to receive a mapping instruction; Conduct mapping in a target site to construct an electronic map of the target site; The region determination module is configured to receive a patrol instruction and determine a patrol region in the electronic map; The picture acquisition module is configured to move in the target site based on the patrol region and acquire pictures at different acquisition points. The picture sending module is configured to send the collection result to the server, so that the server obtains a shelf picture of the target site, identifies a first information carrier in the shelf picture and article information carried by the first information carrier, identifies a second carrier position of a second information carrier in the shelf picture, determines shelf display pictures corresponding to different shelves in the shelf picture based on second carrier positions of different second information carriers, determines that the deployment positions of the second information carriers in different shelves are the same, generates shelf display information based on article information carried by at least one first information carrier in the same shelf display picture, identifies an article in the shelf picture, obtains an article cutout, identifies a cutout position of the article cutout in the shelf picture and a first carrier position of the first information carrier in the shelf picture, establishes a binding relationship between the article cutout and the article information based on the cutout position of the article cutout and the first carrier position of the first information carrier, determines a display position of the corresponding article in the shelf based on the cutout position of the article cutout or the first carrier position of the first information carrier, and generates shelf display information based on the display positions, the article information and the article cutouts of at least one article belonging to the same shelf.

25. An information processing system, characterized by comprising: The mobile robot and the server are included. The mobile robot is configured to perform picture collection in a target site and send a collection result to the server. The target site is provided with at least one shelf, and the at least one shelf is arranged with at least one article and a first information carrier at a respective display position of the at least one article. The first information carrier carries article information of the corresponding article. The at least one shelf is provided with a second information carrier carrying respective shelf information. The deployment positions of the second information carriers in different shelves are the same. The server is configured to acquire a shelf picture collected by the mobile robot, identify a first information carrier in the shelf picture and article information carried by the first information carrier, identify a second information carrier in the shelf picture and a second carrier position of the second information carrier in the shelf picture, determine a shelf display picture corresponding to different shelves in the shelf picture based on second carrier positions of different second information carriers, generate shelf display information based on article information carried by at least one first information carrier in the same shelf display picture, identify an article in the shelf picture, obtain an article cutout, identify a cutout position of the article cutout in the shelf picture and a first carrier position of the first information carrier in the shelf picture, establish a binding relationship between the article cutout and the article information based on the cutout position of the article cutout and the first carrier position of the first information carrier, determine a display position of a corresponding article in the shelf based on the cutout position of the article cutout or the first carrier position of the first information carrier, and generate shelf display information based on the display positions of at least one article in the same shelf, article information, and article cutouts.

26. A computing device, comprising: The information processing device comprises a processing component and a storage component. The storage component stores one or more computer instructions, and the one or more computer instructions are used to be called and executed by the processing component to implement the information processing method according to any one of claims 1-15.

27. An electronic device, comprising: The information processing device comprises a processing component, a display component, and a storage component. The storage component stores one or more computer instructions, and the one or more computer instructions are used to be called and executed by the processing component to implement the information processing method according to any one of claims 16-17.

28. A mobile robot, characterized by The information processing device comprises a processing component, an acquisition component, and a storage component. The storage component stores one or more computer instructions, and the one or more computer instructions are used to be called and executed by the processing component to implement the information processing method according to any one of claims 18-21.

29. A computer storage medium, comprising, A computer program is stored, and the computer program is executed by a computer to implement the information processing method according to any one of claims 1-15.

30. A computer storage medium, comprising, A computer program is stored, and the computer program is executed by a computer to implement the information processing method according to any one of claims 16-17.

31. A computer storage medium, comprising, A computer program is stored, and the computer program is executed by a computer to implement the information processing method according to any one of claims 18-21.

Images