Humanoid robot intelligent control method and system

Abstract

The invention provides an intelligent control method and system for a humanoid robot, and the method comprises the steps: controlling a robot to move in a preset moving channel, and obtaining indoor positioning data and orientation data at the same time; the robot is provided with a forward camera module and a lateral camera module, and image data of a front channel and side goods shelves are continuously collected in the moving process. The system corrects the moving direction of the robot in real time through comprehensive analysis of indoor positioning data, orientation and azimuth data and front image data, and navigation accuracy is ensured. And processing the acquired side shelf images by using a preset commodity image recognition algorithm, recognizing commodity information, matching the commodity information with corresponding indoor positioning data, and finally generating a commodity shelf database containing space positioning information. The multi-dimensional information fusion method not only realizes the accurate navigation of the robot, but also improves the automation level and efficiency of goods shelf commodity management, and provides convenient commodity position query service for users.

Description

Technical Field

[0001] The present application relates to the field of robots, and in particular to an intelligent control method and system for a humanoid robot. Background Art

[0002] Currently, the shelf merchandise management of shopping malls and supermarkets mainly adopts a combination of traditional manual management and electronic management. Manual management includes staff regularly inspecting shelves, replenishing goods, and sorting out the location of goods; electronic management uses technical means such as barcode scanning, electronic price tags, and inventory management systems to input, update, and track product information. In terms of product location, shelf numbering systems, regional zoning labels, and product category labels are mainly used to help users quickly find the products they need. Some large shopping malls have also developed mobile applications that can query specific shelf locations by entering product names.

[0003] However, the existing technology still has many shortcomings. In terms of manual management, due to the wide variety of goods in shopping malls and supermarkets and their frequent changes, it is inevitable that omissions will occur if it relies entirely on manual inspections, resulting in problems such as out-of-stock shelves and chaotic placement of goods. Staff need to invest a lot of time in inspections and sorting, and the labor cost is high. Even with the use of an electronic management system, the timely update of product location information is still a major challenge, because the actual location of the product may not match the system records, especially during promotional activities or after customers pick up and put down the goods.

[0004] The limitations of product positioning are even more obvious. The existing area zoning and shelf numbering systems are often too rough, and customers still need to spend a lot of time looking for specific products in the corresponding area. Although shopping mall mobile applications can provide location information, their accuracy and practicality are relatively limited due to the lack of real-time positioning technology support. In addition, indoor positioning technology is easily interfered in complex shopping mall environments, resulting in inaccurate positioning.

[0005] In terms of product information collection, the traditional barcode scanning method is inefficient and requires scanning one by one to complete information entry. Although electronic price tags are convenient for price updates, they are costly and difficult to display more detailed product information. At the same time, the existing system does not make sufficient use of the analysis of product sales data and fails to fully utilize the value of data to optimize product layout and replenishment strategies. Summary of the invention

[0006] The present application provides a humanoid robot intelligent control method, comprising the following steps: A1, controlling the preset humanoid robot to move in a preset moving channel and obtaining corresponding indoor positioning data and orientation data; A2, when the humanoid robot moves, continuously acquiring front image data and side shelf image data of the humanoid robot; A3, correcting the moving direction of the humanoid robot according to the indoor positioning data, the orientation data and the front image data; A4, matching the corresponding indoor positioning data according to the side shelf image data; A5, identifying and generating information of each product according to the side shelf image data using a preset product image recognition algorithm; A6, combining each product information and the corresponding indoor positioning data to generate product shelf positioning information.

[0007] By adopting the above technical solution, the humanoid robot intelligent control method can realize the precise navigation of the humanoid robot and the automation of commodity shelf management by combining multi-dimensional information such as indoor positioning, orientation data and image recognition. It can not only correct the movement trajectory of the robot in real time, but also synchronously collect and analyze shelf commodity information, and finally generate a commodity information database with precise spatial positioning, thereby improving the efficiency and accuracy of shelf commodity management.

[0008] Optionally, the humanoid robot intelligent control method further comprises the following steps: A7, when a preset query user queries for a product through the humanoid robot, the corresponding product information is matched according to the input query information and defined as related product information; A8, after the query user selects the relevant product information, obtain the product shelf location information corresponding to the relevant product information, and obtain the indoor positioning data defined as the current location data; A9, drawing a corresponding travel route on a preset indoor map image according to the current location data and the product shelf location information to generate a product location map; A10. Display the product location map to the querying user through the humanoid robot.

[0009] By adopting the above technical solution, the humanoid robot intelligent control method can realize an interactive product shopping guide function by drawing a product location map. When a user initiates a query, the system can not only quickly match relevant product information, but also automatically plan and visualize the optimal search route based on the user's current location and the shelf location of the product, thereby improving the shopping experience and allowing users to easily find the desired products, while also improving the service efficiency of shopping malls or supermarkets.

[0010] Optionally, step A3 includes the following steps: A301, determining a corresponding current moving channel on an indoor map image according to the indoor positioning data; A302, determining the direction and orientation data of the corresponding current channel according to the current moving channel; A303, adjusting the orientation of the humanoid robot according to the current channel orientation data until the orientation data is consistent with the current channel orientation data; A304, acquiring the latest front image data and identifying the corresponding left shelf positioning line and right shelf positioning line in the front image data using a preset positioning line recognition algorithm; A305, determining the bottom endpoint pixel position data of the left shelf positioning line in the front image data and defining it as the left endpoint pixel position data; A306, determining a horizontal pixel distance from the left edge of the front image data according to the left endpoint pixel position data and defining the horizontal pixel distance as the left pixel distance; A307, determining the bottom endpoint pixel position data of the right shelf positioning line in the front image data and defining it as the right endpoint pixel position data; A308, determining the horizontal pixel distance from the right edge of the front image data according to the right endpoint pixel position data and defining it as the right pixel distance; A309, calculating the corresponding left and right pixel distance difference according to the difference between the left pixel distance and the right pixel distance; A310, if the left and right pixel distance difference is greater than a preset left deviation threshold, controlling the humanoid robot to translate to the right; A311, if the left and right pixel distance difference is less than a preset right deviation threshold, controlling the humanoid robot to translate to the left; By adopting the above technical solution, the humanoid robot intelligent control method can accurately calculate the relative position relationship between the robot and the left and right shelves by combining indoor map positioning, channel direction recognition and front image analysis, and make real-time deviation adjustments based on the pixel distance difference between the left and right shelf positioning lines, thereby ensuring that the robot can always move in the center of the channel, improving the accuracy and stability of navigation, while also reducing the risk of collision and ensuring the quality of acquiring side shelf image data.

[0011] Optionally, the commodity image recognition algorithm includes the following steps: B1, using a pre-trained label recognition model to identify and determine the label image data of each commodity in the side shelf image data; B2, determining the corresponding label image center pixel point according to each product label image data, and determining the label image center pixel point position data corresponding to the side shelf image data; B3, determining the corresponding label peripheral image acquisition window according to the central pixel position data of each label and the preset peripheral image size frame; B4, acquiring corresponding label peripheral image data from the side shelf image data according to the label peripheral image acquisition window; B5, removing the corresponding product label image data from the label surrounding image data to generate corresponding surrounding independent image data; B6, generating corresponding product label description text according to the product label image data using a preset image text extraction algorithm; B7, generating corresponding product packaging description text based on the surrounding independent image data using an image text extraction algorithm; B8, calculating the corresponding product label text similarity according to the product label description text and the product packaging description text using a preset text similarity algorithm; B9, if the product label text similarity is greater than or equal to a preset similarity threshold, the product label description text is defined as product information.

[0012] By adopting the above technical solution, the humanoid robot intelligent control method can identify the product label image and extract the product image around the product label, and then respectively identify the product label image and the text content in the product image, and ensure that the correspondence between the shelf products and the labels is correct through text similarity comparison, thereby providing reliable data support for automated product management in retail environments.

[0013] Optionally, the commodity image recognition algorithm further comprises the following steps: B10, if the product label text similarity is less than the similarity threshold, the corresponding product label image data is defined as suspected misaligned product label image data; B11, generating a suspected misplaced commodity label information set based on the suspected misplaced commodity label image data and the corresponding side shelf image data and indoor positioning data; B12, sending the suspected misplaced product label information set to the preset control background.

[0014] By adopting the above technical solution, the humanoid robot intelligent control method can detect the mismatch between the product label and the packaging information through the set similarity threshold. It can not only mark the possible misplaced products in time, but also automatically collect relevant image and location data, and feed this information back to the control background in real time, which can effectively improve the accuracy of product display and provide shopping mall managers with the opportunity to correct errors in time, thereby ensuring the efficiency and accuracy of product management.

[0015] Optionally, the text similarity algorithm comprises the following steps: C1, generating corresponding label text phrase data according to the product label description text using a preset word segmentation algorithm; C2, generates corresponding product text phrase data based on the product packaging description text using a word segmentation algorithm; C3, performing deduplication processing on the product text phrase data to generate product text phrase deduplication data; C4, generating a corresponding label text feature vector according to the label text phrase data using a preset feature extraction algorithm; C5, deduplication of product text phrases is used to generate corresponding product text feature vectors using feature extraction algorithm; C6, calculates the corresponding cosine similarity based on the label text feature vector and the product text feature vector and defines it as the product label text similarity.

[0016] By adopting the above technical solution, the humanoid robot intelligent control method can convert the text information on the product label and packaging into a computable feature vector through steps such as word segmentation, deduplication and feature vector extraction, and use cosine similarity to perform similarity calculation. It can not only accurately identify the consistency of product information, but also effectively process text descriptions in different expressions, thereby improving the accuracy of the similarity comparison between products and labels.

[0017] Optionally, the humanoid robot intelligent control method further comprises the following steps: D1, respectively counting the number of occurrences of each phrase corresponding to each phrase in the product text phrase data; D2, the number of phrase occurrences of all phrases was averaged and defined as the average phrase repetition rate; D3, if the average repetition rate of the phrase is less than the preset repetition rate warning threshold, the preset out-of-stock warning information and the corresponding product information and side shelf image data are sent to the control background.

[0018] By adopting the above technical solution, the humanoid robot intelligent control method can analyze the repetition rate of phrases in the commodity description text, and then infer the density of commodities on the shelf. When the phrase repetition rate is lower than the threshold, possible out-of-stock situations can be discovered in time and an alarm can be automatically issued. This intelligent inventory monitoring mechanism based on text analysis provides a real-time inventory warning solution for retail places, which can effectively improve replenishment efficiency.

[0019] The present application also provides a humanoid robot intelligent control system, comprising: A humanoid robot and a control background, wherein the humanoid robot and the control background are data connected; Wherein, the humanoid robot comprises a camera module, a mobile module, a positioning module, an orientation detection module, a query module and a processing control module, and the camera module, the mobile module, the positioning module, the orientation detection module and the query module are respectively data-connected to the processing control module; The humanoid robot intelligent control system further includes a humanoid robot control strategy, including the following steps: E1, controlling the humanoid robot to move in a preset moving channel through the moving module and obtaining corresponding indoor positioning data and orientation data through the positioning module and the orientation detection module respectively; E2, when the humanoid robot moves, continuously acquiring front image data and side shelf image data of the humanoid robot through the camera module; E3, correcting the moving direction of the humanoid robot through the processing control module according to the indoor positioning data, the orientation data and the front image data; E4, matching the corresponding indoor positioning data according to the side shelf image data; E5, identifying and generating information of each product according to the side shelf image data using a preset product image recognition algorithm; E6, combining each product information and the corresponding indoor positioning data to generate product shelf positioning information.

[0020] By adopting the above technical solution, the humanoid robot intelligent control system can realize the interactive product shopping guide function by drawing a product location map. When a user initiates a query, the system can not only quickly match the relevant product information, but also automatically plan and visualize the optimal search route based on the user's current location and the shelf location of the product, thereby improving the shopping experience and allowing users to easily find the required products, while also improving the service efficiency of shopping malls or supermarkets.

[0021] In summary, the present application includes at least one of the following beneficial technical effects: 1. By combining multi-dimensional information such as indoor positioning, orientation data, and image recognition, the precise navigation of humanoid robots and the automation of commodity shelf management can be achieved. It can not only correct the robot's movement trajectory in real time, but also synchronously collect and analyze shelf commodity information, and finally generate a commodity information database with precise spatial positioning, thereby improving the efficiency and accuracy of shelf commodity management.

[0022] 2. The interactive product shopping guide function can be realized by drawing a product location map. When a user initiates a query, the system can not only quickly match the relevant product information, but also automatically plan and visualize the optimal search route based on the user's current location and the location of the product on the shelf, thereby improving the shopping experience and allowing users to easily find the products they need. At the same time, it also improves the service efficiency of shopping malls or supermarkets.

[0023] 3. By combining indoor map positioning, channel direction recognition and front image analysis, the relative position relationship between the robot and the left and right shelves can be accurately calculated, and real-time deviation adjustment can be made based on the pixel distance difference between the left and right shelf positioning lines, ensuring that the robot can always move in the center of the channel, improving navigation accuracy and stability, while also reducing the risk of collision and ensuring the quality of image data acquisition of the side shelves. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 It is a process schematic diagram of a humanoid robot intelligent control method of the present invention.

[0025] Figure 2 It is a principle schematic diagram of an intelligent control system of a humanoid robot according to the present invention. DETAILED DESCRIPTION

[0026] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0027] The embodiments of the present application are further described in detail below in conjunction with the drawings in the specification.

[0028] refer to Figure 1 The present application provides a humanoid robot intelligent control method for patrolling shelves in a shopping mall and recording shelf location information of commodities for user query, comprising the following steps: A1, controlling a preset humanoid robot to move in a preset moving channel and obtaining corresponding indoor positioning data and orientation data; A moving channel is a predetermined road for traffic, for example, an aisle in a supermarket with shelves of various goods on both sides; The indoor positioning data is the positioning data of the humanoid robot in a shopping mall or supermarket, which can be measured and obtained through a pre-set indoor positioning system, for example, through wifi positioning, Bluetooth positioning, UWB positioning and other methods, among which the UWB positioning method is preferred; The orientation data is the orientation data of the humanoid robot, which can be obtained by measuring a direction sensor disposed on the humanoid robot.

[0029] A2, when the humanoid robot moves, continuously acquiring front image data and side shelf image data of the humanoid robot; The front image data is the image data in front of the humanoid robot, including the image of the moving passage in front, which can be acquired by a camera module arranged on the front of the humanoid robot; The side shelf image data is the image data of the side of the humanoid robot, including images of the shelves and goods, which can be acquired by a camera module arranged on the side of the humanoid robot. The side shelf image can be a single-side image or an image of both sides, which can be acquired by setting a camera module on one side or both sides of the humanoid robot.

[0030] A3, correcting the moving direction of the humanoid robot according to the indoor positioning data, the orientation data and the front image data; Since the accuracy of indoor positioning is somewhat insufficient, especially in channels with smaller widths, the moving direction of the humanoid robot may be easily offset due to positioning errors. Therefore, it is necessary to combine the orientation data and the front image data of the humanoid robot to assist in correcting the moving direction of the humanoid robot. On the one hand, it can avoid collisions, and on the other hand, it can stabilize the moving direction of the humanoid robot, further ensuring the quality of the acquisition of the side shelf image data.

[0031] A4, matching the corresponding indoor positioning data according to the side shelf image data; By matching the side shelf image data with the corresponding indoor positioning data, the indoor positioning data when the side shelf image data is acquired can be determined, and the positioning data of the goods in the side shelf image data can be further determined.

[0032] A5, identifying and generating information of each product according to the side shelf image data using a preset product image recognition algorithm; The product image recognition algorithm is a pre-set algorithm used to recognize product information in an image based on the image, for example, it may be an image-based text recognition algorithm, or a pre-trained recognition model, etc.; The product information is the text information of the product in the side shelf image data, which can be stored for user query.

[0033] A6, combining each product information and the corresponding indoor positioning data to generate product shelf positioning information; The product shelf positioning information is the positioning data of the shelf corresponding to the product in the room, which can quickly provide direction guidance through query when the user is looking for the product.

[0034] Through the above steps, the humanoid robot intelligent control method can realize the precise navigation of the humanoid robot and the automation of commodity shelf management by combining multi-dimensional information such as indoor positioning, orientation data and image recognition. It can not only correct the robot's movement trajectory in real time, but also synchronously collect and analyze shelf commodity information, and finally generate a commodity information database with precise spatial positioning, thereby improving the efficiency and accuracy of shelf commodity management.

[0035] Furthermore, the humanoid robot intelligent control method further comprises the following steps: A7, when a preset query user queries for a product through the humanoid robot, the corresponding product information is matched according to the input query information and defined as related product information; The query user is a user who needs to query the location of a product; The query information is the commodity information of the location that the user needs to query, and can be input into the humanoid robot in a variety of ways, for example, input through a touch screen set on the humanoid robot, or input into the robot through voice and then determine the corresponding query information through a language recognition algorithm, etc.; Related product information refers to products that are related to the query information to a certain extent, and a list is provided for users to select and determine the exact products they need to query.

[0036] A8, after the query user selects the relevant product information, obtain the product shelf location information corresponding to the relevant product information, and obtain the indoor positioning data defined as the current location data; The current position data is the current indoor positioning data of the humanoid robot, reflecting the current indoor position information of the querying user.

[0037] A9, drawing a corresponding travel route on a preset indoor map image according to the current location data and the product shelf location information to generate a product location map; The indoor map image is an indoor map that is collected and drawn in advance; The travel route is the route from the location corresponding to the current location data on the indoor map image to the location corresponding to the product shelf positioning information, for the querying customer to follow to obtain the corresponding product; The product location map is an indoor map image with a travel route drawn on it.

[0038] A10, displaying a product location map to the querying user by means of the humanoid robot; The corresponding product location map can be displayed to the inquiring user through a display device arranged on the humanoid robot, and the product location map can also be sent to the mobile device of the inquiring user through an adapted near-field communication method or device.

[0039] Through the above steps, the humanoid robot intelligent control method can realize the interactive product shopping guide function by drawing a product location map. When a user initiates a query, the system can not only quickly match the relevant product information, but also automatically plan and visualize the optimal search route according to the user's current location and the shelf location of the product, thereby improving the shopping experience and allowing users to easily find the required products, while also improving the service efficiency of shopping malls or supermarkets.

[0040] Furthermore, the step A3 comprises the following steps: A301, determining a corresponding current moving channel on an indoor map image according to the indoor positioning data; The current moving channel is the moving channel where the humanoid robot is currently located, which can be determined by matching the indoor positioning data with the indoor map image.

[0041] A302, determining the direction and orientation data of the corresponding current channel according to the current moving channel; The current channel direction and azimuth data is the direction and azimuth data of the current moving channel.

[0042] A303, adjusting the orientation of the humanoid robot according to the current channel orientation data until the orientation data is consistent with the current channel orientation data; By adjusting the orientation of the humanoid robot to be consistent with the current channel direction data, the humanoid robot can move correctly along the current channel direction and avoid deviation in the moving direction.

[0043] A304, acquiring the latest front image data and identifying the corresponding left shelf positioning line and right shelf positioning line in the front image data using a preset positioning line recognition algorithm; The positioning line recognition algorithm is a pre-set algorithm used to identify the positioning line in the image. The positioning line is usually a line drawn on the ground with obvious color difference or texture, which can be easily recognized by the existing algorithm; The left shelf positioning line is a positioning line of the outer edge of the shelf corresponding to the shelf on the left side of the humanoid robot in the front image data; The right shelf positioning line is a positioning line of an outer edge of a shelf corresponding to the setting of the shelf on the right side of the humanoid robot in the front image data.

[0044] A305, determining the pixel position data of the bottom endpoint of the left shelf positioning line in the front image data and defining it as the left endpoint pixel position data; The left end point pixel position data is the pixel position data of the bottom end point of the left shelf positioning line in the front image data.

[0045] A306, determining a horizontal pixel distance from the left edge of the front image data according to the left endpoint pixel position data and defining the horizontal pixel distance as the left pixel distance; The left pixel distance is the horizontal pixel distance from the left endpoint pixel position data to the left edge of the image of the front image data.

[0046] A307, determining the bottom endpoint pixel position data of the right shelf positioning line in the front image data and defining it as the right endpoint pixel position data; The right end point pixel position data is the pixel position data of the bottom end point of the right shelf positioning line in the front image data.

[0047] A308, determining the horizontal pixel distance from the right edge of the front image data according to the right endpoint pixel position data and defining it as the right pixel distance; The right pixel distance is the horizontal pixel distance from the right endpoint pixel position data to the right edge of the image of the front image data.

[0048] A309, calculating the corresponding left and right pixel distance difference according to the difference between the left pixel distance and the right pixel distance; The left-right pixel distance difference is the difference between the left pixel distance and the right pixel distance.

[0049] A310, if the left and right pixel distance difference is greater than a preset left deviation threshold, controlling the humanoid robot to translate to the right; The left deviation threshold is a preset reference value used to determine the deviation of the left side of the humanoid robot. For example, the left deviation threshold can be set to a value close to 0 but greater than 0. Setting a reasonable value can avoid excessive deviation correction of the humanoid robot. When the left and right pixel distance difference is greater than the left deviation threshold, that is, the left pixel distance is greater than the right pixel distance, it means that the humanoid robot is closer to the left shelf and away from the right shelf, and needs to be corrected by translating to the right.

[0050] A311, if the left and right pixel distance difference is less than a preset right deviation threshold, controlling the humanoid robot to translate to the left; The right deviation threshold is a preset reference value used to determine the deviation of the right side of the humanoid robot. For example, the right deviation threshold can be set to a value close to 0 but less than 0. Setting a reasonable value can avoid excessive deviation correction of the humanoid robot. When the left and right pixel distance difference is less than the right bias threshold, that is, the left pixel distance is less than the right pixel distance, it means that the humanoid robot is closer to the right shelf and away from the left shelf.

[0051] Through the above steps, the humanoid robot intelligent control method can accurately calculate the relative position relationship between the robot and the left and right shelves by combining indoor map positioning, channel direction recognition and front image analysis, and make real-time deviation adjustments based on the pixel distance difference between the left and right shelf positioning lines, thereby ensuring that the robot can always move in the center of the channel, improving the accuracy and stability of navigation, while also reducing the risk of collision and ensuring the quality of acquiring side shelf image data.

[0052] Furthermore, the commodity image recognition algorithm comprises the following steps: B1, using a pre-trained label recognition model to identify and determine the label image data of each commodity in the side shelf image data; The label recognition model is a pre-trained model used to recognize product labels in the side shelf image data, and can be generated through training of a large number of product label images; The commodity label image data is image data of the commodity label in the side shelf image data.

[0053] B2, determining the corresponding label image center pixel point according to each product label image data, and determining the label image center pixel point position data corresponding to the side shelf image data; The center pixel point of the label image is the center point of the product label image, which can be calculated and determined according to the pixel coordinates of the four vertices of the rectangular recognition frame corresponding to the product label image data; The label center pixel position data is the pixel position data of the label image center pixel in the side shelf image data.

[0054] B3, determining the corresponding label peripheral image acquisition window according to the central pixel position data of each label and the preset peripheral image size frame; The peripheral image size frame is a preset image acquisition frame used to acquire the product image around the product label. For example, the product label is usually set at the bottom of the shelf where the product is placed. Therefore, the bottom midpoint of the peripheral image size frame and the center pixel point position data of the label can be set to coincide, and the peripheral image size frame is not higher than the height of a single-layer shelf, thereby obtaining the label peripheral image acquisition window; The label peripheral image acquisition window is a window for acquiring a local image of the peripheral area of ​​the label central pixel position data from the side shelf image data.

[0055] B4, acquiring corresponding label peripheral image data from the side shelf image data according to the label peripheral image acquisition window; The label peripheral image data refers to the local image data obtained by the label peripheral image acquisition window on the side shelf image data, that is, the image data around the label center pixel position data.

[0056] B5, removing the corresponding product label image data from the label surrounding image data to generate corresponding surrounding independent image data; The peripheral independent image data is image data generated by removing the product label image data from the label peripheral image data, that is, image data containing only the peripheral product images.

[0057] B6, generating corresponding product label description text according to the product label image data using a preset image text extraction algorithm; The image text extraction algorithm is a pre-set algorithm used to extract text content from images; The product label description text is the text content identified and extracted from the product label image data.

[0058] B7, generating corresponding product packaging description text based on the surrounding independent image data using an image text extraction algorithm; The product packaging description text is the text content identified and extracted from the surrounding independent image data.

[0059] B8, calculating the corresponding product label text similarity according to the product label description text and the product packaging description text using a preset text similarity algorithm; The text similarity algorithm is a pre-set algorithm used to calculate the text similarity between the product label description text and the product packaging description text; The product label text similarity is the text similarity between the product label description text and the product packaging description text.

[0060] B9, if the product label text similarity is greater than or equal to a preset similarity threshold, the product label description text is defined as product information; The similarity threshold is a pre-set reference value used to determine whether the similarity of the product label texts has reached a predetermined level; When the product label text similarity is greater than or equal to the similarity threshold, it can be determined that the product label description text is valid, that is, there is a high correspondence between the surrounding independent image data and the product label image data.

[0061] Through the above steps, the humanoid robot intelligent control method can identify the product label image and extract the product image around the product label, and then identify the product label image and the text content in the product image respectively, and ensure that the correspondence between the shelf products and the labels is correct through text similarity comparison, thereby providing reliable data support for automated product management in retail environments.

[0062] Furthermore, the commodity image recognition algorithm further comprises the following steps: B10, if the product label text similarity is less than the similarity threshold, the corresponding product label image data is defined as suspected misaligned product label image data; The suspected misplaced product label image data is product image data whose product label text similarity is less than a similarity threshold, that is, one of the product or the label may be misplaced.

[0063] B11, generating a suspected misplaced commodity label information set based on the suspected misplaced commodity label image data and the corresponding side shelf image data and indoor positioning data; The suspected misplaced product label information set is a combination of the suspected misplaced product label image data and the corresponding side shelf image data and indoor positioning data.

[0064] B12, sending the suspected misplaced product label information set to the preset control background.

[0065] The control background is a pre-set background that can be monitored and controlled by staff to make corresponding responses.

[0066] Through the above steps, the humanoid robot intelligent control method can detect the mismatch between the product label and the packaging information through the set similarity threshold. It can not only mark the possible misplaced products in time, but also automatically collect relevant image and location data, and feed this information back to the control background in real time, which can effectively improve the accuracy of product display and provide shopping mall managers with the opportunity to correct errors in time, thereby ensuring the efficiency and accuracy of product management.

[0067] Furthermore, the text similarity algorithm comprises the following steps: C1, generating corresponding label text phrase data according to the product label description text using a preset word segmentation algorithm; The word segmentation algorithm is a pre-set algorithm used to segment text. There are many available word segmentation algorithms, which can be selected according to actual needs or effects; The label text phrase data is the phrase data generated by the product label description text through the word segmentation algorithm.

[0068] C2, generates corresponding product text phrase data based on the product packaging description text using a word segmentation algorithm; The commodity text phrase data is phrase data generated by a word segmentation algorithm from the commodity packaging description text.

[0069] C3, performing deduplication processing on the product text phrase data to generate product text phrase deduplication data; Since the product text phrase data is generated based on the product packaging description text, and there are usually multiple identical products in the surrounding independent image data for obtaining the product packaging description text, there will be a lot of repeated text and phrases, which need to be deduplicated; The product text phrase deduplication data is the product text phrase data after deduplication operation.

[0070] C4, generating a corresponding label text feature vector according to the label text phrase data using a preset feature extraction algorithm; The feature extraction algorithm is a pre-set algorithm used to extract feature vectors from label text phrase data; The label text feature vector is a feature vector generated by feature extraction of label text phrase data.

[0071] C5, deduplication of product text phrases is used to generate corresponding product text feature vectors using feature extraction algorithm; The product text feature vector is a feature vector generated by extracting the deduplicated data of product text phrases.

[0072] C6, calculates the corresponding cosine similarity based on the label text feature vector and the product text feature vector and defines it as the product label text similarity; The product label text similarity is the cosine similarity between the label text feature vector and the product text feature vector.

[0073] Through the above steps, the humanoid robot intelligent control method can convert the text information on the product label and packaging into a computable feature vector through steps such as word segmentation, deduplication and feature vector extraction, and use cosine similarity to perform similarity calculation. It can not only accurately identify the consistency of product information, but also effectively process text descriptions in different expressions, thereby improving the accuracy of the similarity comparison between products and labels.

[0074] Furthermore, the humanoid robot intelligent control method further comprises the following steps: D1, respectively counting the number of occurrences of each phrase corresponding to each phrase in the product text phrase data; The number of phrase occurrences is the number of repetitions of each phrase in the commodity text phrase data, which can be determined by statistics.

[0075] D2, the number of phrase occurrences of all phrases was averaged and defined as the average phrase repetition rate; The average phrase repetition rate is the average number of repetitions of all phrases; The average repetition rate of phrases can indirectly reflect the number of identical commodities in the surrounding independent image data corresponding to the commodity text phrase data. For example, if there are three identical commodities, there may be a phrase with three repeated words, that is, the average number of repetitions is close to 3. Therefore, the number of identical commodities can be roughly determined based on the average repetition rate of phrases, and then whether the number of commodities on the shelf is sufficient.

[0076] D3, if the average repetition rate of the phrase is less than the preset repetition rate warning threshold, the preset out-of-stock warning information and the corresponding product information and side shelf image data are sent to the control background; The repetition rate warning threshold is a pre-set reference value used to determine whether the average repetition rate of a phrase is too small; If the average phrase repetition rate is less than the repetition rate warning threshold, it means that the inventory of goods on the shelf may be insufficient, and it is necessary to notify the staff in the control background to replenish the goods in time.

[0077] Through the above steps, the humanoid robot intelligent control method can analyze the repetition rate of phrases in the commodity description text, and then infer the density of commodities on the shelf. When the phrase repetition rate is lower than the threshold, possible out-of-stock situations can be discovered in time and an alarm can be automatically issued. This intelligent inventory monitoring mechanism based on text analysis provides a real-time inventory warning solution for retail places, which can effectively improve replenishment efficiency.

[0078] refer to Figure 2 , the present application also provides a humanoid robot intelligent control system, comprising: A humanoid robot 10 and a control background 20, wherein the humanoid robot 10 and the control background 20 are data connected; The humanoid robot 10 includes a camera module 11, a mobile module 12, a positioning module 13, an orientation detection module 14, a query module 15 and a processing control module 16, wherein the camera module 11, the mobile module 12, the positioning module 13, the orientation detection module 14 and the query module 15 are respectively data-connected to the processing control module 16; The camera module 11 is mainly used to obtain image data in front and on both sides of the humanoid robot 10; The moving module 12 is mainly used to make the humanoid robot 10 move in a controlled manner; The positioning module 13 is mainly used to obtain indoor positioning data of the humanoid robot 10 indoors; The orientation detection module 14 is mainly used to obtain the orientation data of the humanoid robot 10; The query module 15 is mainly used for users to query and provide information feedback on products; The processing control module 16 is mainly used for processing image and text data, and controlling other modules.

[0079] The humanoid robot intelligent control system further includes a humanoid robot control strategy, including the following steps: E1, controlling the humanoid robot to move in a preset moving channel through the moving module 12 and obtaining corresponding indoor positioning data and orientation data through the positioning module 13 and the orientation detection module 14 respectively; E2, when the humanoid robot moves, continuously acquiring the front image data and the side shelf image data of the humanoid robot through the camera module 11; E3, correcting the moving direction of the humanoid robot through the processing control module 16 according to the indoor positioning data, the orientation data and the front image data; E4, matching the corresponding indoor positioning data according to the side shelf image data; E5, identifying and generating information of each product according to the side shelf image data using a preset product image recognition algorithm; E6, combining each product information and the corresponding indoor positioning data to generate product shelf positioning information.

[0080] Through the above steps, the humanoid robot intelligent control system can realize the interactive product shopping guide function by drawing a product location map. When a user initiates a query, the system can not only quickly match the relevant product information, but also automatically plan and visualize the optimal search route according to the user's current location and the shelf location of the product, thereby improving the shopping experience and allowing users to easily find the required products, while also improving the service efficiency of shopping malls or supermarkets.

[0081] The above are all preferred embodiments of the present application, and are not intended to limit the protection scope of the present application. Any feature disclosed in this specification (including the abstract and drawings), unless otherwise stated, can be replaced by other equivalent or alternative features with similar purposes. That is, unless otherwise stated, each feature is only an example of a series of equivalent or similar features.

Claims

1. A humanoid robot intelligent control method, characterized in that: The following steps are involved: A1, controlling the preset humanoid robot to move in a preset moving channel and obtaining corresponding indoor positioning data and orientation data; A2, when the humanoid robot moves, continuously acquiring front image data and side shelf image data of the humanoid robot; A3, correcting the moving direction of the humanoid robot according to the indoor positioning data, the orientation data and the front image data; A4, matching the corresponding indoor positioning data according to the side shelf image data; A5, identifying and generating information of each product according to the side shelf image data using a preset product image recognition algorithm; A6, combining each product information and the corresponding indoor positioning data to generate product shelf positioning information.

2. The humanoid robot intelligent control method according to claim 1, characterized in that: Further comprising the steps of: A7, when a preset query user queries for a product through the humanoid robot, the corresponding product information is matched according to the input query information and defined as related product information; A8, after the query user selects the relevant product information, obtain the product shelf location information corresponding to the relevant product information, and obtain the indoor positioning data defined as the current location data; A9, drawing a corresponding travel route on a preset indoor map image according to the current location data and the product shelf location information to generate a product location map; A10. Display the product location map to the querying user through the humanoid robot.

3. The humanoid robot intelligent control method according to claim 2, characterized in that: Step A3 includes the following steps: A301, determining a corresponding current moving channel on an indoor map image according to the indoor positioning data; A302, determining the direction and orientation data of the corresponding current channel according to the current moving channel; A303, adjusting the orientation of the humanoid robot according to the current channel orientation data until the orientation data is consistent with the current channel orientation data; A304, acquiring the latest front image data and identifying the corresponding left shelf positioning line and right shelf positioning line in the front image data using a preset positioning line recognition algorithm; A305, determining the bottom endpoint pixel position data of the left shelf positioning line in the front image data and defining it as the left endpoint pixel position data; A306, determining a horizontal pixel distance from the left edge of the front image data according to the left endpoint pixel position data and defining it as a left pixel distance; A307, determining the bottom endpoint pixel position data of the right shelf positioning line in the front image data and defining it as the right endpoint pixel position data; A308, determining the horizontal pixel distance from the right edge of the front image data according to the right endpoint pixel position data and defining it as the right pixel distance; A309, calculating the corresponding left and right pixel distance difference according to the difference between the left pixel distance and the right pixel distance; A310, if the left and right pixel distance difference is greater than a preset left deviation threshold, controlling the humanoid robot to translate to the right; A311, if the left and right pixel distance difference is less than a preset right deviation threshold, control the humanoid robot to translate to the left.

4. The humanoid robot intelligent control method according to claim 3, characterized in that: The commodity image recognition algorithm comprises the following steps: B1, using a pre-trained label recognition model to identify and determine the label image data of each commodity in the side shelf image data; B2, determining the corresponding label image center pixel point according to each product label image data, and determining the label image center pixel point position data corresponding to the side shelf image data; B3, determining the corresponding label peripheral image acquisition window according to the central pixel position data of each label and the preset peripheral image size frame; B4, acquiring corresponding label peripheral image data from the side shelf image data according to the label peripheral image acquisition window; B5, removing the corresponding product label image data from the label surrounding image data to generate corresponding surrounding independent image data; B6, generating corresponding product label description text according to the product label image data using a preset image text extraction algorithm; B7, generating corresponding product packaging description text based on the surrounding independent image data using an image text extraction algorithm; B8, calculating the corresponding product label text similarity according to the product label description text and the product packaging description text using a preset text similarity algorithm; B9, if the product label text similarity is greater than or equal to a preset similarity threshold, the product label description text is defined as product information.

5. The humanoid robot intelligent control method according to claim 4, characterized in that: The commodity image recognition algorithm further comprises the following steps: B10, if the product label text similarity is less than the similarity threshold, the corresponding product label image data is defined as suspected misaligned product label image data; B11, generating a suspected misplaced commodity label information set based on the suspected misplaced commodity label image data and the corresponding side shelf image data and indoor positioning data; B12, sending the suspected misplaced product label information set to the preset control background.

6. The humanoid robot intelligent control method according to claim 5, characterized in that: The text similarity algorithm comprises the following steps: C1, generating corresponding label text phrase data according to the product label description text using a preset word segmentation algorithm; C2, generates corresponding product text phrase data based on the product packaging description text using a word segmentation algorithm; C3, performing deduplication processing on the product text phrase data to generate product text phrase deduplication data; C4, generating a corresponding label text feature vector according to the label text phrase data using a preset feature extraction algorithm; C5, deduplication of product text phrases is used to generate corresponding product text feature vectors using feature extraction algorithm; C6, calculates the corresponding cosine similarity based on the label text feature vector and the product text feature vector and defines it as the product label text similarity.

7. The humanoid robot intelligent control method according to claim 6, characterized in that: Further comprising the steps of: D1, respectively counting the number of occurrences of each phrase corresponding to each phrase in the product text phrase data; D2, the number of phrase occurrences of all phrases was averaged and defined as the average phrase repetition rate; D3, if the average repetition rate of the phrase is less than the preset repetition rate warning threshold, the preset out-of-stock warning information and the corresponding product information and side shelf image data are sent to the control background.

8. A humanoid robot intelligent control system, characterized in that: include: A humanoid robot and a control background, wherein the humanoid robot and the control background are data connected; Wherein, the humanoid robot comprises a camera module, a mobile module, a positioning module, an orientation detection module, a query module and a processing control module, and the camera module, the mobile module, the positioning module, the orientation detection module and the query module are respectively data-connected to the processing control module; The humanoid robot intelligent control system further includes a humanoid robot control strategy, including the following steps: E1, controlling the humanoid robot to move in a preset moving channel through the moving module and obtaining corresponding indoor positioning data and orientation data through the positioning module and the orientation detection module respectively; E2, when the humanoid robot moves, continuously acquiring front image data and side shelf image data of the humanoid robot through the camera module; E3, correcting the moving direction of the humanoid robot through the processing control module according to the indoor positioning data, the orientation data and the front image data; E4, matching the corresponding indoor positioning data according to the side shelf image data; E5, identifying and generating information of each product according to the side shelf image data using a preset product image recognition algorithm; E6, combining each product information and the corresponding indoor positioning data to generate product shelf positioning information.

Images