An intelligent control method and system for a humanoid robot

Abstract

The present application provides an intelligent control method and system for a humanoid robot. By controlling the robot to move within a preset moving channel, indoor positioning data and orientation data are acquired simultaneously. The robot is equipped with forward and lateral camera modules, and continuously collects image data of the front channel and the lateral shelves during movement. The system corrects the moving direction of the robot in real time through comprehensive analysis of the indoor positioning data, orientation data, and front image data to ensure navigation accuracy. For the collected lateral shelf images, a preset commodity image recognition algorithm is used for processing to identify commodity information and match it with the corresponding indoor positioning data, and finally a commodity shelf database containing spatial positioning information is generated. This method of multi-dimensional information fusion not only realizes the precise navigation of the robot, but also improves the automation level and efficiency of shelf commodity management, providing users with a convenient commodity location query service.

Description

Technical Field

[0001] This application relates to the field of robots, and in particular to an intelligent control method and system for humanoid robots. Background Art

[0002] Currently, the management of shelf commodities in shopping malls and supermarkets mainly adopts a combination of traditional manual management and electronic management. Manual management includes regular inspections of shelves by staff, replenishment, and rearrangement of commodity positions; electronic management realizes the entry, update, and tracking of commodity information through technical means such as barcode scanning, electronic price tags, and inventory management systems. In terms of commodity positioning, the shelf numbering system, regional zoning signs, commodity category labels, etc. are mainly used to help users quickly find the required commodities. Some large shopping malls have also developed mobile applications, through which the specific shelf location can be queried by entering the commodity name.

[0003] However, there are still many deficiencies in the existing technologies. In terms of manual management, due to the large variety and frequent changes of commodities in shopping malls and supermarkets, relying solely on manual inspections will inevitably lead to oversights, resulting in problems such as out-of-stock shelves and chaotic commodity placement. Staff need to invest a lot of time in inspections and arrangements, and the labor cost is relatively high. Even with the use of an electronic management system, the timely update of commodity location information is still a major challenge because the actual location of the commodity may not match the system record, especially during promotional activities or after customers pick up and place commodities.

[0004] The limitations of the commodity positioning function are even more obvious. The existing regional zoning and shelf numbering systems are often too rough, and customers still need to spend a lot of time looking for specific commodities in the corresponding areas. Although the mobile applications of shopping malls can provide location information, due to the lack of real-time positioning technology support, their accuracy and practicality are relatively limited. In addition, indoor positioning technology is easily interfered in a complex shopping mall environment, resulting in inaccurate positioning.

[0005] In terms of commodity information collection, the traditional barcode scanning method is inefficient and requires one-by-one scanning to complete information entry. Although electronic price tags are convenient for price updates, they are costly and difficult to display more detailed commodity information. At the same time, the existing systems do not make full use of the analysis of commodity sales data and fail to fully utilize the data value to optimize commodity layout and replenishment strategies. Summary of the Invention

[0006] This application provides an intelligent control method for a humanoid robot, including the following steps:

[0007] A1. Control the preset humanoid robot to move within a preset moving channel and obtain corresponding indoor positioning data and orientation data;

[0008] A2, when the humanoid robot moves, continuously acquiring front image data and side shelf image data of the humanoid robot;

[0009] A3, correcting the moving direction of the humanoid robot according to the indoor positioning data, the orientation data and the front image data;

[0010] A4, matching the corresponding indoor positioning data according to the side shelf image data;

[0011] A5, identifying and generating information of each product according to the side shelf image data using a preset product image recognition algorithm;

[0012] A6, combining each product information and the corresponding indoor positioning data to generate product shelf positioning information.

[0013] By adopting the above technical solution, the humanoid robot intelligent control method can realize the precise navigation of the humanoid robot and the automation of commodity shelf management by combining multi-dimensional information such as indoor positioning, orientation data and image recognition. It can not only correct the movement trajectory of the robot in real time, but also synchronously collect and analyze shelf commodity information, and finally generate a commodity information database with precise spatial positioning, thereby improving the efficiency and accuracy of shelf commodity management.

[0014] Optionally, the humanoid robot intelligent control method further comprises the following steps:

[0015] A7, when a preset query user queries for a product through the humanoid robot, the corresponding product information is matched according to the input query information and defined as related product information;

[0016] A8, after the query user selects the relevant product information, obtain the product shelf location information corresponding to the relevant product information, and obtain the indoor positioning data defined as the current location data;

[0017] A9, drawing a corresponding travel route on a preset indoor map image according to the current location data and the product shelf location information to generate a product location map;

[0018] A10. Display the product location map to the querying user through the humanoid robot.

[0019] By adopting the above technical solution, the humanoid robot intelligent control method can realize an interactive product shopping guide function by drawing a product location map. When a user initiates a query, the system can not only quickly match relevant product information, but also automatically plan and visualize the optimal search route based on the user's current location and the shelf location of the product, thereby improving the shopping experience and allowing users to easily find the desired products, while also improving the service efficiency of shopping malls or supermarkets.

[0020] Optionally, step A3 includes the following steps:

[0021] A301. Determine the corresponding current moving passage according to the indoor positioning data in the indoor map image;

[0022] A302. Determine the corresponding current passage orientation data according to the current moving passage;

[0023] A303. Adjust the orientation of the humanoid robot according to the current passage orientation data so that the orientation data is consistent with the current passage orientation data;

[0024] A304. Obtain the latest front image data and identify the corresponding left shelf positioning line and right shelf positioning line in the front image data by using a preset positioning line recognition algorithm;

[0025] A305. Determine the bottom endpoint pixel position data of the left shelf positioning line in the front image data and define it as the left endpoint pixel position data;

[0026] A306. Determine the horizontal pixel distance from the left endpoint pixel position data to the left edge of the front image data and define it as the left pixel distance;

[0027] A307. Determine the bottom endpoint pixel position data of the right shelf positioning line in the front image data and define it as the right endpoint pixel position data;

[0028] A308. Determine the horizontal pixel distance from the right endpoint pixel position data to the right edge of the front image data and define it as the right pixel distance;

[0029] A309. Calculate the difference between the left pixel distance and the right pixel distance to calculate the corresponding left - right pixel distance difference;

[0030] A310. If the left - right pixel distance difference is greater than a preset left - deviation threshold, control the humanoid robot to translate to the right;

[0031] A311. If the left - right pixel distance difference is less than a preset right - deviation threshold, control the humanoid robot to translate to the left;

[0032] By adopting the above - mentioned technical solution, the intelligent control method of the humanoid robot can, by combining indoor map positioning, passage orientation recognition and front - image analysis, accurately calculate the relative position relationship between the robot and the left and right shelves, and perform real - time deviation adjustment according to the pixel distance difference of the left and right shelf positioning lines, ensuring that the robot can always travel in the center of the passage, improving the accuracy and stability of navigation, reducing the collision risk at the same time, and also ensuring the acquisition quality of the side - shelf image data.

[0033] Optionally, the commodity image recognition algorithm includes the following steps:

[0034] B1. Use a pre-trained label recognition model to identify and determine each commodity label image data in the side shelf image data;

[0035] B2. Determine the corresponding label image center pixel points according to each commodity label image data, and determine the position data of the label center pixel points corresponding to the label image center pixel points in the side shelf image data;

[0036] B3. Determine the corresponding label peripheral image acquisition window according to each label center pixel point position data and a preset peripheral image size frame;

[0037] B4. Acquire the corresponding label peripheral image data from the side shelf image data according to the label peripheral image acquisition window;

[0038] B5. Remove the corresponding commodity label image data from the label peripheral image data to generate the corresponding peripheral independent image data;

[0039] B6. Generate the corresponding commodity label description text according to the commodity label image data with a preset image text extraction algorithm;

[0040] B7. Generate the corresponding commodity packaging description text according to the peripheral independent image data with an image text extraction algorithm;

[0041] B8. Calculate the corresponding commodity label text similarity according to the commodity label description text and the commodity packaging description text with a preset text similarity algorithm;

[0042] B9. If the commodity label text similarity is greater than or equal to the preset similarity threshold, define the commodity label description text as commodity information.

[0043] By adopting the above technical solution, the intelligent control method of the humanoid robot can identify the commodity label image, extract the commodity image around the commodity label, then respectively identify the text content in the commodity label image and the commodity image, and ensure the correct corresponding relationship between the shelf commodity and the label through text similarity comparison, providing reliable data support for the automated commodity management in the retail environment.

[0044] Optionally, the commodity image recognition algorithm further includes the following steps:

[0045] B10. If the commodity label text similarity is less than the similarity threshold, define the corresponding commodity label image data as suspected misaligned commodity label image data;

[0046] B11. Generate a set of suspected misaligned product label information based on the suspected misaligned product label image data, the corresponding side shelf image data, and the indoor positioning data combination.

[0047] B12. Send the set of suspected misaligned product label information to a preset control background.

[0048] By adopting the above technical solution, the intelligent control method of the humanoid robot can detect the mismatch between the product label and the packaging information through a set similarity threshold. It can not only mark the products that may be misaligned in a timely manner, but also automatically collect relevant image and position data, and feedback this information to the control background in real time. This can effectively improve the accuracy of product display, provide an opportunity for mall managers to correct errors in a timely manner, and thus ensure the efficiency and accuracy of product management.

[0049] Optionally, the text similarity algorithm includes the following steps:

[0050] C1. Generate corresponding label text phrase data according to the product label description text with a preset word segmentation algorithm.

[0051] C2. Generate corresponding product text phrase data according to the product packaging description text with a word segmentation algorithm.

[0052] C3. Perform duplicate removal processing on the product text phrase data to generate duplicate-removed product text phrase data.

[0053] C4. Generate corresponding label text feature vectors according to the label text phrase data with a preset feature extraction algorithm.

[0054] C5. Generate corresponding product text feature vectors from the duplicate-removed product text phrase data with a feature extraction algorithm.

[0055] C6. Calculate the corresponding cosine similarity based on the label text feature vectors and the product text feature vectors and define it as the product label text similarity.

[0056] By adopting the above technical solution, the intelligent control method of the humanoid robot can, through steps such as word segmentation, duplicate removal, and feature vector extraction, convert the text information on the product label and packaging into computable feature vectors, and use cosine similarity for similarity calculation. It can not only accurately identify the consistency of product information, but also effectively process text descriptions in different expression forms, improving the accuracy of comparing the similarity between products and labels.

[0057] Optionally, the intelligent control method of the humanoid robot further includes the following steps:

[0058] D1. Statistically count the occurrence times of each phrase corresponding to the phrases in the product text phrase data respectively.

[0059] D2. Calculate the average number of occurrences of each phrase and define it as the average phrase repetition rate;

[0060] D3. If the average phrase repetition rate is less than the preset repetition rate warning threshold, send the preset out-of-stock warning information, the corresponding product information, and the side shelf image data to the control background.

[0061] By adopting the above technical solution, the intelligent control method of the humanoid robot can infer the density of products on the shelf by analyzing the repetition rate of phrases in the product description text. When the phrase repetition rate is lower than the threshold, it can timely detect possible out-of-stock situations and automatically alarm. This intelligent inventory monitoring mechanism based on text analysis provides a real-time inventory warning solution for retail places and can effectively improve the replenishment efficiency.

[0062] This application also provides an intelligent control system for a humanoid robot, including:

[0063] A humanoid robot and a control background, where the humanoid robot and the control background are connected by data;

[0064] Among them, the humanoid robot includes a camera module, a movement module, a positioning module, an orientation detection module, a query module, and a processing and control module. The camera module, the movement module, the positioning module, the orientation detection module, and the query module are respectively connected to the processing and control module by data;

[0065] The intelligent control system of the humanoid robot further includes a humanoid robot control strategy, including the following steps:

[0066] E1. Control the humanoid robot to move within a preset movement channel through the movement module, and respectively obtain the corresponding indoor positioning data and orientation data through the positioning module and the orientation detection module;

[0067] E2. When the humanoid robot is moving, continuously obtain the front image data and side shelf image data of the humanoid robot through the camera module;

[0068] E3. Correct the movement direction of the humanoid robot through the processing and control module according to the indoor positioning data, the orientation data, and the front image data;

[0069] E4. Match the corresponding indoor positioning data according to the side shelf image data;

[0070] E5. Identify and generate each product information according to the side shelf image data with a preset product image recognition algorithm;

[0071] E6. Combine each product information and the corresponding indoor positioning data to generate product shelf positioning information.

[0072] By adopting the above technical solution, the intelligent control system of the humanoid robot can realize the interactive product shopping guide function by drawing a product positioning map. When a user initiates a query, the system can not only quickly match relevant product information, but also automatically plan and visually display the optimal item-finding route according to the user's current location and the location of the product on the shelf, improving the shopping experience, enabling the user to conveniently find the required products, and at the same time improving the service efficiency of the mall or supermarket.

[0073] In summary, the present application includes at least one of the following beneficial technical effects:

[0074] 1. By combining multi-dimensional information such as indoor positioning, orientation data, and image recognition, the precise navigation of the humanoid robot and the automation of product shelf management are realized. It can not only correct the moving trajectory of the robot in real time, but also synchronously collect and analyze the information of the products on the shelf, and finally generate a product information database with precise spatial positioning, improving the efficiency and accuracy of product shelf management.

[0075] 2. By drawing a product positioning map, the interactive product shopping guide function can be realized. When a user initiates a query, the system can not only quickly match relevant product information, but also automatically plan and visually display the optimal item-finding route according to the user's current location and the location of the product on the shelf, improving the shopping experience, enabling the user to conveniently find the required products, and at the same time improving the service efficiency of the mall or supermarket.

[0076] 3. By combining indoor map positioning, aisle direction recognition, and front image analysis, the relative position relationship between the robot and the left and right shelves can be accurately calculated, and real-time deviation adjustment can be performed according to the pixel distance difference of the positioning lines of the left and right shelves, ensuring that the robot can always travel in the center of the aisle, improving the accuracy and stability of navigation, reducing the collision risk, and ensuring the acquisition quality of the side shelf image data. BRIEF DESCRIPTION OF THE DRAWINGS

[0077] Figure 1 is a schematic process diagram of a method for intelligent control of a humanoid robot according to the present invention.

[0078] Figure 2 is a schematic principle diagram of an intelligent control system of a humanoid robot according to the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0079] In order to make the objectives, technical solutions, and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0080] The embodiments of the present application will be further described in detail below with reference to the accompanying drawings of the specification.

[0081] Referring Figure 1 , the present application provides an intelligent control method for a humanoid robot, which is used to patrol the shelves in a shopping mall and record the shelf positioning information of commodities for users to query, including the following steps:

[0082] A1. Control the preset humanoid robot to move within a preset moving passage and obtain corresponding indoor positioning data and orientation data;

[0083] The moving passage is a pre-determined passable road. For example, the aisles in a supermarket, where there are shelves with various commodities on both sides;

[0084] The indoor positioning data is the positioning data of the humanoid robot within a shopping mall or supermarket, which can be measured and obtained through a pre-set indoor positioning system. For example, it can be obtained through methods such as wifi positioning, Bluetooth positioning, UWB positioning, etc. Among them, the UWB positioning method is preferred;

[0085] The orientation data is the orientation data of the humanoid robot, which can be measured and obtained through a direction sensor set on the humanoid robot.

[0086] A2. Continuously obtain the front image data and side shelf image data of the humanoid robot while the humanoid robot is moving;

[0087] The front image data is the image data directly in front of the humanoid robot, which contains the image of the moving passage directly in front and can be collected through a camera module set on the front of the humanoid robot;

[0088] The side shelf image data is the image data on the side of the humanoid robot, which contains the images of the shelves and commodities and can be collected through a camera module set on the side of the humanoid robot. The side shelf image can be a single-sided image or images on both the left and right sides, and can be obtained by setting camera modules on one side or both sides of the humanoid robot.

[0089] A3. Correct the moving direction of the humanoid robot according to the indoor positioning data, orientation data and front image data;

[0090] Due to the certain deficiency in the accuracy of indoor positioning, especially in a passage with a small width, it is easy for the moving direction of the humanoid robot to deviate due to positioning errors. Therefore, it is necessary to combine the orientation data and front image data of the humanoid robot to assist in correcting the moving direction of the humanoid robot. On the one hand, it can avoid collisions, and on the other hand, it can stabilize the moving direction of the humanoid robot and further ensure the acquisition quality of the side shelf image data.

[0091] A4. Match the corresponding indoor positioning data according to the side shelf image data;

[0092] By matching the side shelf image data with the corresponding indoor positioning data, the indoor positioning data when the side shelf image data is acquired can be determined, and further the positioning data of the goods in the side shelf image data can be determined.

[0093] A5. Identify and generate each commodity information according to the side shelf image data with a preset commodity image recognition algorithm;

[0094] The commodity image recognition algorithm is a preset algorithm for identifying the commodity information in the image according to the image. For example, it can be an image-based text recognition algorithm or a pre-trained recognition model, etc.;

[0095] The commodity information is the text information of the goods in the side shelf image data, which can be stored for users to query.

[0096] A6. Combine each commodity information and the corresponding indoor positioning data to generate commodity shelf positioning information;

[0097] The commodity shelf positioning information is the positioning data of the shelf corresponding to the commodity in the room. When the user is looking for a commodity, it can quickly give orientation guidance by querying.

[0098] Through the above steps, the intelligent control method of the humanoid robot can realize the precise navigation of the humanoid robot and the automation of commodity shelf management by combining multi-dimensional information such as indoor positioning, orientation data, and image recognition. It can not only correct the moving trajectory of the robot in real time, but also synchronously collect and analyze the commodity information on the shelf, and finally generate a commodity information database with precise spatial positioning, improving the efficiency and accuracy of shelf commodity management.

[0099] Furthermore, the intelligent control method of the humanoid robot further includes the following steps:

[0100] A7. When a preset query user queries a commodity through the humanoid robot, match the corresponding commodity information according to the input query information and define it as relevant commodity information;

[0101] The query user is the user who needs to query the location of the commodity;

[0102] The query information is the commodity information whose location the user needs to query, and can be input into the humanoid robot in various ways. For example, it can be input through the touch screen set on the humanoid robot, or input into the robot by voice and then the corresponding query information is determined through a speech recognition algorithm, etc.;

[0103] Related product information refers to products that are related to the query information to a certain extent, and a list is provided for users to select and determine the exact products they need to query.

[0104] A8, after the query user selects the relevant product information, obtain the product shelf location information corresponding to the relevant product information, and obtain the indoor positioning data defined as the current location data;

[0105] The current position data is the current indoor positioning data of the humanoid robot, reflecting the current indoor position information of the querying user.

[0106] A9, drawing a corresponding travel route on a preset indoor map image according to the current location data and the product shelf location information to generate a product location map;

[0107] The indoor map image is an indoor map that is collected and drawn in advance;

[0108] The travel route is the route from the location corresponding to the current location data on the indoor map image to the location corresponding to the product shelf positioning information, for the querying customer to follow to obtain the corresponding product;

[0109] The product location map is an indoor map image with a travel route drawn on it.

[0110] A10, displaying a product location map to the querying user by means of the humanoid robot;

[0111] The corresponding product location map can be displayed to the inquiring user through a display device arranged on the humanoid robot, and the product location map can also be sent to the mobile device of the inquiring user through an adapted near-field communication method or device.

[0112] Through the above steps, the humanoid robot intelligent control method can realize the interactive product shopping guide function by drawing a product location map. When a user initiates a query, the system can not only quickly match the relevant product information, but also automatically plan and visualize the optimal search route according to the user's current location and the shelf location of the product, thereby improving the shopping experience and allowing users to easily find the required products, while also improving the service efficiency of shopping malls or supermarkets.

[0113] Furthermore, the step A3 comprises the following steps:

[0114] A301, determining a corresponding current moving channel on an indoor map image according to the indoor positioning data;

[0115] The current moving channel is the moving channel where the humanoid robot is currently located, which can be determined by matching the indoor positioning data with the indoor map image.

[0116] A302, determining the direction and orientation data of the corresponding current channel according to the current moving channel;

[0117] The current channel direction azimuth data is the azimuth data of the direction of the current moving channel.

[0118] A303. Adjust the orientation of the humanoid robot according to the current channel direction azimuth data so that the orientation azimuth data is consistent with the current channel direction data;

[0119] By adjusting the orientation of the humanoid robot to be consistent with the current channel direction data, the humanoid robot can move correctly along the current channel direction, avoiding deviation of the moving direction.

[0120] A304. Obtain the latest front image data and identify the corresponding left shelf positioning line and right shelf positioning line in the front image data with a preset positioning line recognition algorithm;

[0121] The positioning line recognition algorithm is a preset algorithm for recognizing the positioning line in the image. The positioning line is usually a line drawn on the ground with obvious color difference or texture and can be easily recognized by existing algorithms;

[0122] The left shelf positioning line is the positioning line of the outer edge of the shelf corresponding to the setting of the shelf on the left side of the humanoid robot in the front image data;

[0123] The right shelf positioning line is the positioning line of the outer edge of the shelf corresponding to the setting of the shelf on the right side of the humanoid robot in the front image data.

[0124] A305. Determine the pixel position data of the bottom endpoint of the left shelf positioning line in the front image data and define it as the left endpoint pixel position data;

[0125] The left endpoint pixel position data is the pixel position data of the bottom endpoint of the left shelf positioning line in the front image data.

[0126] A306. Determine the horizontal pixel distance from the left endpoint pixel position data to the left edge of the front image data and define it as the left pixel distance;

[0127] The left pixel distance is the horizontal pixel distance from the left endpoint pixel position data to the left edge of the front image data.

[0128] A307. Determine the pixel position data of the bottom endpoint of the right shelf positioning line in the front image data and define it as the right endpoint pixel position data;

[0129] The right endpoint pixel position data is the pixel position data of the bottom endpoint of the right shelf positioning line in the front image data.

[0130] A308. Determine the horizontal pixel distance from the right endpoint pixel position data to the right edge of the front image data and define it as the right pixel distance.

[0131] The right pixel distance is the horizontal pixel distance from the right endpoint pixel position data to the right edge of the front image data.

[0132] A309. Calculate the corresponding left - right pixel distance difference by taking the difference between the left pixel distance and the right pixel distance.

[0133] The left - right pixel distance difference is the difference between the left pixel distance and the right pixel distance.

[0134] A310. If the left - right pixel distance difference is greater than the preset left - deviation threshold, control the humanoid robot to translate to the right.

[0135] The left - deviation threshold is a preset reference value used to judge the deviation on the left side of the humanoid robot. For example, the left - deviation threshold can be set as a value close to 0 but greater than 0. Setting a reasonable value can prevent the humanoid robot from over - correcting.

[0136] When the left - right pixel distance difference is greater than the left - deviation threshold, that is, the left pixel distance is greater than the right pixel distance, it means that the humanoid robot is closer to the left shelf and farther from the right shelf, and needs to be corrected by translating to the right.

[0137] A311. If the left - right pixel distance difference is less than the preset right - deviation threshold, control the humanoid robot to translate to the left.

[0138] The right - deviation threshold is a preset reference value used to judge the deviation on the right side of the humanoid robot. For example, the right - deviation threshold can be set as a value close to 0 but less than 0. Setting a reasonable value can prevent the humanoid robot from over - correcting.

[0139] When the left - right pixel distance difference is less than the right - deviation threshold, that is, the left pixel distance is less than the right pixel distance, it means that the humanoid robot is closer to the right shelf and farther from the left shelf.

[0140] Through the above steps, the intelligent control method of the humanoid robot can accurately calculate the relative position relationship between the robot and the left and right shelves by combining indoor map positioning, aisle direction recognition, and front - image analysis, and perform real - time deviation adjustment according to the pixel distance difference of the left and right shelf positioning lines, ensuring that the robot can always move in the center of the aisle, improving the accuracy and stability of navigation, reducing the collision risk, and also ensuring the acquisition quality of the side - shelf image data.

[0141] Further, the commodity image recognition algorithm includes the following steps:

[0142] B1. Use a pre-trained label recognition model to identify and determine each piece of product label image data in the side shelf image data;

[0143] The label recognition model is a pre-trained model for identifying product labels in side shelf image data and can be generated by training with images of a large number of product labels;

[0144] The product label image data is the image data of the product labels in the side shelf image data.

[0145] B2. Determine the corresponding label image center pixel points according to each piece of product label image data, and determine the position data of the label center pixel points corresponding to the label image center pixel points in the side shelf image data;

[0146] The label image center pixel point is the center point of the product label image and can be calculated and determined according to the pixel coordinates of the four vertices of the rectangular recognition frame corresponding to the product label image data;

[0147] The position data of the label center pixel points is the pixel position data of the label image center pixel points in the side shelf image data.

[0148] B3. Determine the corresponding label peripheral image acquisition window according to the position data of each label center pixel point and a preset peripheral image size frame;

[0149] The peripheral image size frame is a preset image acquisition frame for acquiring product images around the product label. For example, since the product label is usually set at the lower position of the shelf where the product is placed, the midpoint of the bottom edge of the peripheral image size frame can be set to coincide with the position data of the label center pixel point, and the peripheral image size frame is not higher than the height of a single-layer shelf, so as to obtain the label peripheral image acquisition window;

[0150] The label peripheral image acquisition window is a window for acquiring local image data around the position data of the label center pixel points in the side shelf image data.

[0151] B4. Acquire the corresponding label peripheral image data according to the label peripheral image acquisition window in the side shelf image data;

[0152] The label peripheral image data is the local image data acquired by the label peripheral image acquisition window in the side shelf image data, that is, the image data around the position data of the label center pixel points.

[0153] B5. Remove the corresponding product label image data from the label peripheral image data to generate the corresponding peripheral independent image data;

[0154] The peripheral independent image data is the image data generated after removing the product label image data from the label peripheral image data, that is, the image data only containing the peripheral product images.

[0155] B6. Generate a corresponding product label description text according to the product label image data by using a preset image text extraction algorithm;

[0156] The image text extraction algorithm is a preset algorithm for extracting text content in an image;

[0157] The product label description text is the text content identified and extracted from the product label image data.

[0158] B7. Generate a corresponding product packaging description text according to the peripheral independent image data by using the image text extraction algorithm;

[0159] The product packaging description text is the text content identified and extracted from the peripheral independent image data.

[0160] B8. Calculate the corresponding product label text similarity according to the product label description text and the product packaging description text by using a preset text similarity algorithm;

[0161] The text similarity algorithm is a preset algorithm for calculating the text similarity between the product label description text and the product packaging description text;

[0162] The product label text similarity is the text similarity between the product label description text and the product packaging description text.

[0163] B9. If the product label text similarity is greater than or equal to a preset similarity threshold, define the product label description text as product information;

[0164] The similarity threshold is a preset reference value for judging whether the product label text similarity has reached a predetermined degree;

[0165] When the product label text similarity is greater than or equal to the similarity threshold, it can be determined that the product label description text is valid, that is, there is a high corresponding relationship between the peripheral independent image data and the product label image data.

[0166] Through the above steps, the intelligent control method of the humanoid robot can identify the product label image, extract the product images around the product label, and then respectively identify the text content in the product label image and the product images, and ensure the correct corresponding relationship between the shelf products and the labels through text similarity comparison, providing reliable data support for automated product management in the retail environment.

[0167] Further, the product image recognition algorithm further includes the following steps:

[0168] B10. If the similarity of the product label text is less than the similarity threshold, define the corresponding product label image data as suspected misaligned product label image data;

[0169] The suspected misaligned product label image data is the product image data with the similarity of the product label text less than the similarity threshold, that is, there may be a situation where one of the product or the label is misplaced.

[0170] B11. Generate a suspected misaligned product label information set based on the suspected misaligned product label image data, the corresponding side shelf image data, and the indoor positioning data;

[0171] The suspected misaligned product label information set is a data combination of the suspected misaligned product label image data, the corresponding side shelf image data, and the indoor positioning data.

[0172] B12. Send the suspected misaligned product label information set to a preset control background.

[0173] The control background is a preset background that can be monitored and controlled by staff to make corresponding responses.

[0174] Through the above steps, the intelligent control method of the humanoid robot can detect the mismatch between the product label and the packaging information through the set similarity threshold. It can not only mark the products that may be misaligned in time, but also automatically collect relevant image and position data, and feedback this information to the control background in real time. This can effectively improve the accuracy of product display, provide an opportunity for mall managers to correct errors in time, and thus ensure the efficiency and accuracy of product management.

[0175] Further, the text similarity algorithm includes the following steps:

[0176] C1. Generate corresponding label text phrase data according to the product label description text with a preset word segmentation algorithm;

[0177] The word segmentation algorithm is a preset algorithm for word segmentation of text. There are various available word segmentation algorithms, which can be selected according to actual needs or effects;

[0178] The label text phrase data is the phrase data generated by the product label description text through the word segmentation algorithm.

[0179] C2. Generate corresponding product text phrase data according to the product packaging description text with the word segmentation algorithm;

[0180] The product text phrase data is the phrase data generated by the product packaging description text through the word segmentation algorithm.

[0181] C3. Perform duplicate removal processing on the product text phrase data to generate product text phrase duplicate removal data;

[0182] Since the commodity text phrase data is generated based on the commodity packaging description text, and there are generally multiple identical commodities in the surrounding independent image data for obtaining the commodity packaging description text, there will be a lot of duplicate text and phrases, and a deduplication operation is required;

[0183] The deduplicated data of commodity text phrases is the commodity text phrase data after the deduplication operation.

[0184] C4. Generate the corresponding label text feature vector according to the label text phrase data with a preset feature extraction algorithm;

[0185] The feature extraction algorithm is a preset algorithm for extracting the feature vector in the label text phrase data;

[0186] The label text feature vector is the feature vector generated by feature extraction from the label text phrase data.

[0187] C5. Generate the corresponding commodity text feature vector from the deduplicated data of commodity text phrases with the feature extraction algorithm;

[0188] The commodity text feature vector is the feature vector generated by feature extraction from the deduplicated data of commodity text phrases.

[0189] C6. Calculate the corresponding cosine similarity based on the label text feature vector and the commodity text feature vector and define it as the commodity-label text similarity;

[0190] The commodity-label text similarity is the cosine similarity between the label text feature vector and the commodity text feature vector.

[0191] Through the above steps, the intelligent control method of the humanoid robot can convert the commodity label and the text information on the packaging into computable feature vectors through steps such as word segmentation, deduplication, and feature vector extraction, and use the cosine similarity for similarity calculation. It can not only accurately identify the consistency of commodity information, but also effectively process text descriptions in different expression forms, improving the accuracy of comparing the similarity between commodities and labels.

[0192] Furthermore, the intelligent control method of the humanoid robot further includes the following steps:

[0193] D1. Statistically count the occurrence times of each phrase corresponding to the commodity text phrase data respectively;

[0194] The occurrence times of the phrase are the repetition times of each phrase in the commodity text phrase data, which can be statistically determined.

[0195] D2. Calculate the average value of the occurrence times of all phrases and define it as the average phrase repetition rate;

[0196] The average phrase repetition rate is the average number of repetitions of all phrases;

[0197] The average phrase repetition rate can indirectly reflect the quantity of the same products in the surrounding independent image data corresponding to the product text phrases. For example, if there are 3 identical products, there may be phrases with 3 repeated words, that is, the average number of repetitions is close to 3. Therefore, the number of the same products can be roughly judged according to the average phrase repetition rate, and then it can be judged whether the quantity of products on the shelf is sufficient.

[0198] D3, if the average phrase repetition rate is less than the preset repetition rate warning threshold, send the preset out-of-stock warning information, the corresponding product information, and the side shelf image data to the control background;

[0199] The repetition rate warning threshold is a preset reference value used to judge whether the average phrase repetition rate is too small;

[0200] If the average phrase repetition rate is less than the repetition rate warning threshold, it means that the stock of products on the shelf may be insufficient, and the staff in the control background needs to be notified to replenish the shelves in time.

[0201] Through the above steps, the intelligent control method of the humanoid robot can infer the density of products on the shelf by analyzing the repetition rate of phrases in the product description text. When the phrase repetition rate is lower than the threshold, it can timely detect possible out-of-stock situations and automatically alarm. This intelligent inventory monitoring mechanism based on text analysis provides a real-time inventory warning solution for retail places and can effectively improve the replenishment efficiency.

[0202] Reference Figure 2 , this application also provides an intelligent control system for a humanoid robot, including:

[0203] A humanoid robot 10 and a control background 20, and the humanoid robot 10 and the control background 20 are data-connected;

[0204] Among them, the humanoid robot 10 includes a camera module 11, a moving module 12, a positioning module 13, an orientation detection module 14, a query module 15, and a processing and control module 16. The camera module 11, the moving module 12, the positioning module 13, the orientation detection module 14, and the query module 15 are respectively data-connected to the processing and control module 16;

[0205] The camera module 11 is mainly used to obtain the image data in front of and on both sides of the humanoid robot 10;

[0206] The moving module 12 is mainly used to make the humanoid robot 10 displace controllably;

[0207] The positioning module 13 is mainly used to obtain the indoor positioning data of the humanoid robot 10 indoors;

[0208] The orientation detection module 14 is mainly used to obtain the orientation data of the humanoid robot 10;

[0209] The query module 15 is mainly used for users to query products and provide information feedback;

[0210] The processing and control module 16 is mainly used for processing data such as images and texts, and controlling other modules.

[0211] The intelligent control system of the humanoid robot further includes a control strategy for the humanoid robot, including the following steps:

[0212] E1, controlling the humanoid robot to move within a preset moving path through the moving module 12, and respectively obtaining corresponding indoor positioning data and orientation data through the positioning module 13 and the orientation detection module 14;

[0213] E2, continuously obtaining the front image data of the humanoid robot and the side shelf image data through the camera module 11 when the humanoid robot is moving;

[0214] E3, correcting the moving direction of the humanoid robot through the processing and control module 16 according to the indoor positioning data, orientation data and front image data;

[0215] E4, matching the corresponding indoor positioning data according to the side shelf image data;

[0216] E5, identifying and generating each product information according to the side shelf image data with a preset product image recognition algorithm;

[0217] E6, combining each product information and the corresponding indoor positioning data to generate product shelf positioning information.

[0218] Through the above steps, the intelligent control system of the humanoid robot can realize the interactive product shopping guide function by drawing a product positioning map. When the user initiates a query, the system can not only quickly match the relevant product information, but also automatically plan and visually display the optimal product search route according to the user's current position and the position of the product on the shelf, improving the shopping experience, enabling the user to conveniently find the required products, and at the same time improving the service efficiency of the mall or supermarket.

[0219] The above are all preferred embodiments of the present application, and do not limit the protection scope of the present application accordingly. Any feature disclosed in this specification (including the abstract and drawings), unless specifically described, can be replaced by other equivalent or similar-purpose alternative features. That is, unless specifically described, each feature is only an example of a series of equivalent or similar features.

Claims

1. An intelligent control method for a humanoid robot, characterized in that Including the following steps: A1. Control the preset humanoid robot to move within a preset moving channel and obtain corresponding indoor positioning data and orientation data; A2. Continuously obtain the front image data and side shelf image data of the humanoid robot while the humanoid robot is moving; A3. Correct the moving direction of the humanoid robot according to the indoor positioning data, orientation data, and front image data; A4. Match the corresponding indoor positioning data according to the side shelf image data; A5. Identify and generate various product information according to the side shelf image data by using a preset product image recognition algorithm; A6. Combine the various product information and the corresponding indoor positioning data to generate product shelf positioning information; Wherein, the product image recognition algorithm includes the following steps: B1. Identify and determine each product label image data in the side shelf image data by using a pre-trained label recognition model; B2. Determine the corresponding label image center pixel points according to each product label image data, and determine the label center pixel point position data corresponding to the label image center pixel points in the side shelf image data; B3. Determine the corresponding label peripheral image acquisition window according to each label center pixel point position data and a preset peripheral image size frame; B4. Acquire the corresponding label peripheral image data from the side shelf image data according to the label peripheral image acquisition window; B5. Remove the corresponding product label image data from the label peripheral image data to generate corresponding peripheral independent image data; B6. Generate the corresponding product label description text according to the product label image data by using a preset image text extraction algorithm; B7. Generate the corresponding product packaging description text according to the peripheral independent image data by using an image text extraction algorithm; B8. Calculate the corresponding product label text similarity according to the product label description text and the product packaging description text by using a preset text similarity algorithm; B9. If the product label text similarity is greater than or equal to a preset similarity threshold, define the product label description text as product information.

2. The intelligent control method for a humanoid robot according to claim 1, wherein Further including the following steps: A7. When a preset query user queries a product through the humanoid robot, match the corresponding product information according to the input query information and define it as relevant product information; A8. After the query user selects the relevant product information, obtain the product shelf positioning information corresponding to the relevant product information, and obtain the indoor positioning data and define it as the current position data; A9. Draw a corresponding travel route on a preset indoor map image according to the current position data and the product shelf positioning information to generate a product positioning map; A10. Display the product positioning map to the query user through the humanoid robot.

3. The intelligent control method for a humanoid robot according to claim 2, characterized in that, Step A3 includes the following steps: A301. Determine the corresponding current moving channel in the indoor map image according to the indoor positioning data; A302. Determine the corresponding current channel orientation data according to the current moving channel; A303. Adjust the orientation of the humanoid robot according to the current channel orientation data so that the orientation data is consistent with the current channel orientation data; A304. Obtain the latest front image data and identify the corresponding left shelf positioning line and right shelf positioning line in the front image data using a preset positioning line recognition algorithm; A305. Determine the bottom endpoint pixel position data of the left shelf positioning line in the front image data and define it as the left endpoint pixel position data; A306. Determine the horizontal pixel distance from the left endpoint pixel position data to the left edge of the front image data and define it as the left pixel distance; A307. Determine the bottom endpoint pixel position data of the right shelf positioning line in the front image data and define it as the right endpoint pixel position data; A308. Determine the horizontal pixel distance from the right endpoint pixel position data to the right edge of the front image data and define it as the right pixel distance; A309. Calculate the difference between the left pixel distance and the right pixel distance to obtain the corresponding left - right pixel distance difference; A310. If the left - right pixel distance difference is greater than a preset left - deviation threshold, control the humanoid robot to translate to the right; A311. If the left - right pixel distance difference is less than a preset right - deviation threshold, control the humanoid robot to translate to the left.

4. The intelligent control method for a humanoid robot according to claim 3, characterized in that, The commodity image recognition algorithm further includes the following steps: B10. If the similarity of the commodity label text is less than the similarity threshold, define the corresponding commodity label image data as suspected misaligned commodity label image data; B11. Generate a suspected misaligned commodity label information set based on the suspected misaligned commodity label image data, the corresponding side - shelf image data, and indoor positioning data; B12. Send the suspected misaligned commodity label information set to a preset control background.

5. The intelligent control method for a humanoid robot according to claim 4, characterized in that The text similarity algorithm includes the following steps: C1. Generate corresponding label text phrase data from the commodity label description text using a preset word - segmentation algorithm; C2. Generate corresponding commodity text phrase data from the commodity packaging description text using a word - segmentation algorithm; C3. Perform deduplication processing on the commodity text phrase data to generate commodity text phrase deduplication data; C4. Generate corresponding label text feature vectors from the label text phrase data using a preset feature extraction algorithm; C5. Generate corresponding commodity text feature vectors from the commodity text phrase deduplication data using a feature extraction algorithm; C6. Calculate the corresponding cosine similarity based on the label text feature vectors and the commodity text feature vectors and define it as the commodity label text similarity.

6. The intelligent control method for a humanoid robot according to claim 5, wherein, It further includes the following steps: D1. Statistically count the occurrence times of each phrase in the commodity text phrase data respectively; D2. Calculate the average value of the occurrence times of all phrases and define it as the phrase average repetition rate; D3. If the phrase average repetition rate is less than a preset repetition rate warning threshold, send a preset out - of - stock warning message, the corresponding commodity information, and side - shelf image data to the control background.

7. An intelligent control system for a humanoid robot, characterized in that, It includes: A humanoid robot and a control background, and the humanoid robot is data - connected to the control background; Among them, the humanoid robot includes a camera module, a movement module, a positioning module, an orientation detection module, a query module, and a processing and control module. The camera module, the movement module, the positioning module, the orientation detection module, and the query module are respectively connected to the processing and control module through data links; The intelligent control system of the humanoid robot further includes a humanoid robot control strategy, which includes the following steps: E1, control the humanoid robot to move within a preset movement path through the movement module, and respectively obtain corresponding indoor positioning data and orientation data through the positioning module and the orientation detection module; E2, when the humanoid robot is moving, continuously obtain the front image data and the side shelf image data of the humanoid robot through the camera module; E3, correct the movement direction of the humanoid robot through the processing and control module according to the indoor positioning data, the orientation data, and the front image data; E4, match the corresponding indoor positioning data according to the side shelf image data; E5, generate various commodity information by identifying with a preset commodity image recognition algorithm according to the side shelf image data; E6, combine the various commodity information and the corresponding indoor positioning data to generate commodity shelf positioning information; Among them, the commodity image recognition algorithm includes the following steps: B1, identify and determine the respective commodity label image data in the side shelf image data with a pre-trained label recognition model; B2, determine the corresponding label image center pixel points according to the respective commodity label image data, and determine the label center pixel point position data corresponding to the label image center pixel points in the side shelf image data; B3, determine the corresponding label surrounding image acquisition window according to the respective label center pixel point position data and a preset surrounding image size frame; B4, obtain the corresponding label surrounding image data from the side shelf image data according to the label surrounding image acquisition window; B5, remove the corresponding commodity label image data from the label surrounding image data to generate the corresponding surrounding independent image data; B6, generate the corresponding commodity label description text with a preset image text extraction algorithm according to the commodity label image data; B7, generate the corresponding commodity packaging description text with an image text extraction algorithm according to the surrounding independent image data; B8, calculate the corresponding commodity label text similarity with a preset text similarity algorithm according to the commodity label description text and the commodity packaging description text; B9, if the commodity label text similarity is greater than or equal to a preset similarity threshold, define the commodity label description text as commodity information.

Images