Target user determination method, apparatus and electronic device based on robot vision

Abstract

The application relates to a target user determination method, device and electronic device based on robot vision, wherein the target user determination method based on robot vision comprises the following steps: acquiring position data between a plurality of users and a robot within a preset time; the robot is used for acquiring position information of the user according to a vision module of the robot, and the position data is determined according to the position information of the user and position information of the robot; determining corresponding weighting coefficients of the plurality of users according to the position data; sorting the plurality of users according to the weighting coefficients to obtain a sorting result, and determining a target user according to the sorting result. Through the application, the problem that a robot is difficult to find a VIP user among a plurality of users is solved, and user experience is improved.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence, and in particular to methods, apparatus and electronic devices for target user identification based on robot vision. Background Technology

[0002] Service robots are mainly used to communicate with users. Generally, after the user inputs a command via voice, the robot will perform a specific action or communicate with the user based on the command.

[0003] When many users surround the robot and give it voice commands, the robot will look at the VIP users out of courtesy and answer their questions and follow their commands first to enhance their experience. However, it is difficult for the robot to find the VIP users among the many users.

[0004] There is currently no effective solution to the problem that robots have difficulty finding VIP users among a large number of users in related technologies. Summary of the Invention

[0005] This embodiment provides a target user identification method, apparatus, and electronic device based on robot vision to solve the problem in related technologies where it is difficult for robots to find VIP users among a large number of users.

[0006] Firstly, this embodiment provides a target user determination method based on robot vision, the method comprising:

[0007] The robot acquires position data between multiple users and the robot within a preset time period; the robot is used to acquire the user's position information based on the robot's vision module, and the position data is determined based on the user's position information and the robot's position information.

[0008] Based on the location data, weighting coefficients are determined for multiple users;

[0009] The multiple users are sorted according to the weighting coefficients to obtain a sorting result, and the target user is determined based on the sorting result.

[0010] In some embodiments, acquiring location data between multiple users and the robot within a preset time period includes:

[0011] A preset frequency for collecting location information is established, and location information of multiple users is collected according to the preset frequency.

[0012] Based on the robot's location information and the location information of multiple users, position data between multiple users and the robot is determined; the position data includes the distance between the user and the robot and the angle between the user and the robot.

[0013] A user identifier for a plurality of users is determined, and an information set for the plurality of users is constructed; wherein, the user identifier is unique, and the information set includes the user identifier and the location information of the user corresponding to the user identifier.

[0014] In some embodiments, determining the location data between the multiple users and the robot based on the robot's location information and the location information of the multiple users includes:

[0015] Construct a coordinate system with the robot's position coordinates as the origin;

[0016] The user's location information includes the user's coordinates in a coordinate system constructed with the robot's position coordinates as the origin;

[0017] Based on the robot's position coordinates and the user's coordinates, determine the position data between the user and the robot.

[0018] In some embodiments, determining the position data between the user and the robot based on the robot's position coordinates and the user's coordinates includes:

[0019] Based on the user's coordinates, determine the distance between the user and the robot;

[0020] The angle between the user and the robot is determined based on the vector corresponding to the user's coordinates and the preset direction vector of the robot.

[0021] The distance between the user and the robot and the angle between the user and the robot are determined as the positional data between the user and the robot.

[0022] In some embodiments, determining the weighting coefficients corresponding to multiple users based on the location data includes:

[0023] Based on the information set of multiple users, determine the distance between the multiple users and the robot at a preset time point;

[0024] The distances between the multiple users and the robot are sorted to obtain the sorting results;

[0025] Based on the sorting results, a first weighting coefficient is determined for each of the users.

[0026] In some embodiments, determining the weighting coefficients corresponding to the multiple users based on the location data further includes:

[0027] Based on the information sets of the multiple users, determine the direction vectors of the multiple users;

[0028] The angle between the user and the robot is determined based on the preset direction vector of the robot and the direction vector of the user;

[0029] The angles of the multiple users relative to the robot are sorted to obtain a sorting result;

[0030] Based on the sorting results, a second weighting coefficient is determined for each of the users;

[0031] The weighting coefficients for the multiple users are determined based on the first weighting coefficient and the second weighting coefficient.

[0032] In some embodiments, the step of sorting the multiple users according to the weighting coefficient to obtain a sorting result, and determining the target user based on the sorting result, includes:

[0033] Obtain the weighted coefficients corresponding to multiple users at different times within a preset time period, and sum the weighted coefficients corresponding to multiple users at different times within the preset time period to obtain the total weighted coefficient;

[0034] The total weighted coefficients are sorted to obtain a sorting result, and the target user is determined based on the sorting result.

[0035] In some embodiments, the step of sorting the multiple users according to the weighting coefficient to obtain a sorting result, and determining the target user based on the sorting result, further includes:

[0036] When there are multiple target users, obtain the first weighting coefficient and the second weighting coefficient for each target user.

[0037] The ratio of the first weighting coefficient and the second weighting coefficient in the weighting coefficient is preset, wherein the ratio of the second weighting coefficient is greater than the ratio of the first weighting coefficient;

[0038] Calculate the weighting coefficients for multiple target users according to a preset ratio;

[0039] Calculate the total weighted coefficient of the multiple target users based on their weighted coefficients;

[0040] The multiple total weighted coefficients are sorted to obtain a sorting result, and the target user is re-determined based on the sorting result.

[0041] Secondly, this embodiment provides a target user determination device based on robot vision. The device includes: an acquisition module, a processing module, and a determination module. The acquisition module is used to acquire position data between multiple users and the robot within a preset time period. The robot is used to acquire the user's position information according to the robot's vision module, and the position data is determined based on the user's position information and the robot's position information.

[0042] The processing module is used to determine the weighting coefficients corresponding to multiple users based on the location data;

[0043] The determination module is used to sort multiple users according to the weighting coefficients, obtain a sorting result, and determine the target user based on the sorting result.

[0044] Thirdly, this embodiment provides an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the target user determination method based on robot vision described in the first aspect above.

[0045] Compared with related technologies, the target user determination method based on robot vision provided in this embodiment obtains the location information of multiple users within a preset time period; determines the weighting coefficients corresponding to multiple users based on the location information; sorts the users according to the weighting coefficients to obtain the sorting result; and then determines the target user based on the sorting result. This helps the robot to find the target user more accurately and conveniently and serve the target user.

[0046] Details of one or more embodiments of this application are set forth in the following drawings and description to make other features, objects and advantages of this application more readily apparent. Attached Figure Description

[0047] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments of this application and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0048] Figure 1 This is a hardware structure block diagram of the terminal of the target user determination method based on robot vision in this embodiment;

[0049] Figure 2 This is a schematic diagram of a scenario in this embodiment;

[0050] Figure 3 This is a flowchart of the target user determination method based on robot vision in this embodiment;

[0051] Figure 4This is a flowchart of the target user determination method based on robot vision in this specific embodiment;

[0052] Figure 5 This is a preprocessing flowchart of a specific embodiment;

[0053] Figure 6 Example diagram for identifying the target user in this specific embodiment;

[0054] Figure 7 This is a structural block diagram of the target user determination device based on robot vision in this embodiment. Detailed Implementation

[0055] To better understand the purpose, technical solution, and advantages of this application, the application is described and illustrated below in conjunction with the accompanying drawings and embodiments.

[0056] Unless otherwise defined, the technical or scientific terms used in this application shall have the general meaning as understood by one of ordinary skill in the art to which this application pertains. Words such as “a,” “an,” “an,” “the,” “the,” and “these,” used in this application, do not indicate quantitative limitation and may be singular or plural. The terms “comprising,” “including,” “having,” and any variations thereof used in this application are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or device that comprises a series of steps or modules (units) is not limited to the listed steps or modules (units) but may include steps or modules (units) not listed, or may include other steps or modules (units) inherent to such processes, methods, products, or devices. The terms “connected,” “linked,” and “coupled,” used in this application, are not limited to physical or mechanical connections but may include electrical connections, whether direct or indirect. The term “multiple” used in this application refers to two or more. The "and / or" operator describes the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: A alone, A and B simultaneously, and B alone. Typically, the character " / " indicates that the objects before and after it are in an "or" relationship. The terms "first," "second," and "third," etc., used in this application are merely for distinguishing similar objects and do not represent a specific ordering of the objects.

[0057] The method embodiments provided in this example can be executed on a terminal, computer, or similar computing device. For example, it can run on a terminal. Figure 1 This is a hardware structure block diagram of the terminal for the target user determination method based on robot vision in this embodiment. (See diagram for example.) Figure 1 As shown, a terminal may include one or more ( Figure 1Only one is shown in the diagram. A processor 102 and a memory 104 for storing data are also included. The processor 102 may be, but is not limited to, a microprocessor (MCU) or a programmable logic device (FPGA). The terminal may also include a transmission device 106 for communication functions and an input / output device 108. Those skilled in the art will understand that… Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the terminal described above. For example, the terminal may also include components that are larger than... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown are illustrated.

[0058] The memory 104 can be used to store computer programs, such as application software programs and modules, like the computer program corresponding to the target user determination method based on robot vision in this embodiment. The processor 102 executes various functional applications and data processing by running the computer programs stored in the memory 104, thereby implementing the above-described method. The memory 104 may include high-speed random access memory and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0059] The transmission device 106 is used to receive or send data via a network. This network includes a wireless network provided by the terminal's communication provider. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 can be a Radio Frequency (RF) module used for wireless communication with the Internet.

[0060] In this embodiment, Figure 2 This is a schematic diagram of a scenario in this embodiment, such as... Figure 2 As shown, when multiple users enter a venue that requires a robot to provide guidance or answer questions, they will choose to input voice commands to a nearby robot. The robot will then identify the location information of the multiple users using its vision module and determine who the VIP user is, i.e., the target user. Once the robot identifies the target user, it will perform specific actions or communicate with the user based on the user's voice input.

[0061] This embodiment provides a target user determination method based on robot vision. Figure 3 This is a flowchart of the target user determination method based on robot vision in this embodiment, as shown below. Figure 3 As shown, the process includes the following steps:

[0062] Step S310: Obtain position data between multiple users and the robot within a preset time period; the robot uses its vision module to obtain the user's position information, and the position data is determined based on the user's position information and the robot's position information.

[0063] Specifically, when multiple users enter the venue, the robot obtains the location information of the multiple users based on the robot's vision module. Subsequently, the processor determines, based on the location information of the multiple users and the robot's location information, to acquire multiple location data between the multiple users and the robot within a preset time.

[0064] Step S320: Determine the weighting coefficients for multiple users based on the location data.

[0065] Specifically, after the processor acquires multiple positional data between multiple users and the robot entering the venue, it determines the weighting coefficients corresponding to the multiple users based on the positional data.

[0066] Step S330: Sort multiple users according to weighting coefficients to obtain sorting results, and determine the target user based on the sorting results.

[0067] Specifically, after sorting the weighted coefficients, the processor determines the target user corresponding to each weighted coefficient based on the sorting result.

[0068] Through steps S310 to S330, after multiple users enter the venue, the robot acquires their location information using its vision module. Then, the processor, based on the user and robot location information, determines how many location data points between the users and the robot to be acquired within a preset timeframe. After acquiring this data, the processor determines weighting coefficients for each user and sorts them. Based on this sorting, it identifies the target user corresponding to each weighting coefficient. This process helps the robot more accurately and conveniently locate and serve the target user, improving the user experience.

[0069] In some embodiments, step S310 includes steps S311 to S313.

[0070] Step S311: Preset the frequency of location information collection, and collect the location information of multiple users according to the frequency.

[0071] Specifically, the processor collects location information from multiple users according to a preset collection frequency.

[0072] Step S312: Based on the robot's position information and the position information of multiple users, determine the position data between the multiple users and the robot; the position data includes the distance between the users and the robot and the angle between the users and the robot.

[0073] Specifically, the processor determines the distance and angle between the user and the robot based on the robot's position information and the user's position information obtained through the robot's vision module. The distance and angle between the user and the robot are the position data.

[0074] Step S313: Determine the user identifiers of multiple users and construct an information set for multiple users; wherein, the user identifier is unique, and the information set includes the user identifier and the location information of the user corresponding to the user identifier.

[0075] Specifically, after the processor obtains the user's location information, it determines the unique user identifier of multiple users around the robot, and constructs a user information set based on the user identifier and the user's location information.

[0076] Through steps S311 to S313, the processor collects the location information of multiple users according to a preset collection frequency, determines the unique user identifiers of multiple users around the robot, and constructs a user information set based on the user identifiers and user location information. Based on the robot's location information and the user's location information obtained through the robot's vision module, the distance and angle between the user and the robot are determined; these distance and angle constitute the location data. By collecting user location information at a preset frequency, determining user identifiers, and constructing a user information set, while simultaneously determining the angle and distance between the user and the robot based on the robot's and user's location information, the accuracy of subsequently determining the weighted coefficients corresponding to the user is improved, further enhancing the accuracy of identifying the target user.

[0077] In some embodiments, step S312 includes steps S301 to S303.

[0078] Step S301: Construct a coordinate system with the robot's position coordinates as the origin.

[0079] Specifically, the processor uses the robot's position coordinates as the origin and sets the x-axis, y-axis, positive x-axis direction, and positive y-axis direction of the coordinate system based on the robot's position coordinates, thereby constructing a coordinate system with the robot's position coordinates as the origin.

[0080] Step S302: The user's location information includes the user's coordinates in a coordinate system constructed with the robot's position coordinates as the origin.

[0081] Specifically, once a user enters the venue, a coordinate system is constructed with the robot's position coordinates as the origin to obtain the user's coordinates, which are the user's location information.

[0082] Step S303: Determine the position data between the user and the robot based on the robot's position coordinates and the user's coordinates.

[0083] Specifically, the processor determines the position data between the user and the robot based on the robot's position coordinates and the user's position information.

[0084] Through steps S301 to S303, the processor constructs a coordinate system with the robot's position coordinates as the origin. When the user enters the site, the processor determines the distance and angle between the user and the robot based on the robot's position information and the user's position information obtained through the robot's vision module. The distance and angle between the user and the robot constitute the position data. Constructing a coordinate system with the robot's position coordinates as the origin helps to further improve the accuracy of obtaining user position data. At the same time, obtaining the distance and angle between the user and the robot, and thus obtaining position data, helps to further improve the accuracy of subsequently determining the user's corresponding weighting coefficient, and further improves the accuracy of identifying the target user.

[0085] In some of these embodiments, step S303 includes steps S3031 to S3033.

[0086] Step S3031: Determine the distance between the user and the robot based on the user's coordinates.

[0087] Specifically, the processor constructs a coordinate system with the robot's position coordinates as the origin. When the user enters the site, the processor obtains the user's coordinates and determines the straight-line distance between the user and the robot based on the user's coordinates and the robot's origin position coordinates.

[0088] Step S3032: Determine the angle between the user and the robot based on the vector corresponding to the user's coordinates and the preset robot direction vector.

[0089] Specifically, the processor calculates the angle between the user and the robot based on the vector pointing from the origin to the user's coordinates and the preset robot direction vector.

[0090] Step S3033: Determine the distance between the user and the robot and the angle between the user and the robot as the position data between the user and the robot.

[0091] Specifically, the processor determines the distance and angle between the user and the robot as the position data between the user and the robot.

[0092] Through steps S3031 to S3033, the processor determines the straight-line distance between the user and the robot based on the robot's position coordinates and the user's position information; it then calculates the angle between the user and the robot based on the vector pointing from the origin to the user's coordinates and the preset robot direction vector; finally, it uses the acquired distance and angle between the user and the robot to determine the position data between them. Obtaining the distance and angle between the user and the robot to obtain position data helps improve the accuracy of subsequently determining the user's corresponding weighting coefficient, and further improves the accuracy of identifying the target user.

[0093] In some embodiments, step S320 includes steps S321 to S323.

[0094] Step S321: Based on the information set of multiple users, determine the distance between multiple users and the robot at a preset time point.

[0095] Specifically, the processor determines multiple distances between the same user and the robot at multiple moments within a preset time period based on information sets from multiple users, including the user's corresponding user identifier and the user's location information.

[0096] Step S322: Sort the distances between multiple users and the robot to obtain the sorting results.

[0097] Specifically, the processor sorts the distances between multiple users and the robot at multiple times, and then obtains the sorting results of the distances between the users.

[0098] Step S323: Determine the first weighting coefficients for multiple users based on the sorting results.

[0099] Specifically, the processor presets multiple first weighting coefficients and determines the first weighting coefficients corresponding to multiple users based on the sorting results.

[0100] Through steps S321 to S323, the processor determines multiple distances between the same user and the robot at multiple moments within a preset time period based on information sets from multiple users. The processor then sorts the distances between the multiple users and the robot at these multiple moments to obtain a sorting result for the distances corresponding to each user. Multiple first weighting coefficients are preset, and the first weighting coefficients corresponding to the distances between the users and the robot are determined based on the sorting result. By collecting user location information at multiple moments within a preset time period, sorting the user location information at these multiple moments, and then determining the first weighting coefficients corresponding to different users based on the sorting result, the accuracy of identifying target users is improved.

[0101] In some embodiments, step S320 further includes steps S324 to S328.

[0102] Step S324: Determine the direction vectors of multiple users based on the information set of multiple users.

[0103] Specifically, the processor uses the location information in the user's information set, which is specifically the user's coordinates, as the starting point of the vector, with the robot's position coordinates (i.e., the origin) as the starting point and the user's coordinates as the ending point, to obtain the user's direction vector.

[0104] Step S325: Determine the angle between the user and the robot based on the preset robot direction vector and user direction vector.

[0105] Specifically, the processor presets the robot's direction vector, which is a unit vector, and obtains the angle between the user and the robot based on the robot's direction vector and the user's direction vector.

[0106] Step S326: Sort the angles of multiple users and the robot to obtain the sorting results.

[0107] Step S327: Determine the second weighting coefficients for multiple users based on the sorting results.

[0108] Specifically, the processor sorts the angles of multiple users and the robot according to preset rules to obtain a sorting result; and determines the second weighting coefficient corresponding to each user based on the sorting result.

[0109] Step S328: Determine the weighting coefficients for multiple users based on the first weighting coefficient and the second weighting coefficient.

[0110] Specifically, the processor calculates the weighting coefficients of multiple users at the same time based on the preset ratio of the first weighting coefficient and the second weighting coefficient, wherein the ratio of the first weighting coefficient and the second weighting coefficient is 1:1 by default.

[0111] Through steps S324 to S328, the processor obtains the user's direction vector based on the location information in the user's information set, specifically the user's coordinates. Using the robot's coordinates (origin) as the starting point and the user's coordinates as the ending point, the processor calculates the user's direction vector. The processor presets the robot's direction vector (a unit vector) and, based on the robot's and user's direction vectors, obtains the angle between the user and the robot. Multiple angles between users and the robot are sorted according to preset rules to obtain a sorting result. A second weighting coefficient is determined for each user based on the sorting result. The processor calculates the total weighting coefficient for multiple users based on the preset ratio of the first and second weighting coefficients. By obtaining the user's first weighting coefficient related to distance and the second weighting coefficient related to angle, the processor obtains the user's weighting coefficient at the same time, thereby improving the accuracy of identifying the target user.

[0112] In some embodiments, step S330 includes steps S331 to S332.

[0113] Step S331: Obtain the weighted coefficients corresponding to multiple users at different times within a preset time period, and sum the weighted coefficients corresponding to multiple users at different times within the preset time period to obtain the total weighted coefficient.

[0114] Specifically, the processor acquires the weighted coefficients corresponding to multiple users at different times within a preset time period, and accumulates the weighted coefficients corresponding to the same user to obtain the total weighted coefficient of multiple users.

[0115] Step S332: Sort the total weighted coefficients to obtain the sorting results, and determine the target users based on the sorting results.

[0116] Specifically, after the processor obtains the total weighted coefficients of multiple users, it sorts the total weighted coefficients according to preset rules to obtain the sorting results, and determines the target user based on the sorting results.

[0117] Through steps S331 to S332, the processor acquires the weighted coefficients corresponding to multiple users at different times within a preset time period, and accumulates the weighted coefficients corresponding to the same user to obtain the total weighted coefficient of multiple users. Then, the total weighted coefficients are sorted according to preset rules to obtain the sorting result, and the target user is determined based on the sorting result. By acquiring the total weighted coefficients of users at different times within a preset time period, sorting them, and obtaining the sorting result, the target user is finally confirmed, which helps to further improve the accuracy of determining the target user.

[0118] In some embodiments, step S330 further includes steps S333 to S337.

[0119] Step S333: When there are multiple target users, obtain the first weighting coefficient and the second weighting coefficient of each target user.

[0120] Specifically, when the processor determines that there are multiple target users based on the total weighting coefficient of multiple users, it obtains the first weighting coefficient and the second weighting coefficient of the multiple target users within a preset time period.

[0121] Step S334: Preset the ratio of the first weighting coefficient and the second weighting coefficient in the weighting coefficient, wherein the ratio of the second weighting coefficient is greater than the ratio of the first weighting coefficient.

[0122] Step S335: Calculate the weighting coefficients for multiple target users according to the preset ratio.

[0123] Specifically, the processor presets the ratio of the first weighting coefficient and the second weighting coefficient in the total weighting coefficient, and then calculates the weighting coefficients for multiple target users respectively.

[0124] Step S336: Calculate the total weighting coefficient of multiple target users based on their weighting coefficients.

[0125] Specifically, the processor calculates the total weighting coefficient of multiple target users based on the weighting coefficients of target users at different times within a preset time period and the preset proportion of the weighting coefficients.

[0126] Step S337: Sort the multiple total weighted coefficients to obtain the sorting results, and redetermine the target users based on the sorting results.

[0127] Specifically, the processor sorts the total weighted coefficients of the multiple target users obtained again, obtains the reordered result, and then redetermines the target users based on the reordered result.

[0128] Through steps S333 to S337, when the processor determines that there are multiple target users based on the total weighted coefficient of multiple users, it obtains the first weighted coefficient and the second weighted coefficient of each target user within a preset time period; it presets the ratio of the first weighted coefficient and the second weighted coefficient in the total weighted coefficient, and then calculates the weighted coefficient of each target user; based on the weighted coefficient of the target users at different times within the preset time period and the preset ratio of the weighted coefficient, it calculates the total weighted coefficient of the multiple target users; and it sorts the newly obtained total weighted coefficient of the multiple target users to obtain a re-sorted result, and then redetermines the target users based on the re-sorted result. When there are multiple target users, by resetting the ratio of the first weighted coefficient and the second weighted coefficient, and then re-sorting the total weighted coefficient of the multiple target users, and then determining new target users based on the re-sorted result, it is beneficial to further improve the accuracy and efficiency of determining target users, and further improve the user experience.

[0129] The present embodiment will now be described and illustrated through preferred embodiments.

[0130] Figure 4 This is a flowchart of the target user determination method based on robot vision in this specific embodiment, as shown below. Figure 4 As shown, the target user determination method based on robot vision includes the following steps:

[0131] Step S410, Information preprocessing.

[0132] Specifically, a coordinate system is constructed with the robot's location as the origin. For example, the robot's left and right directions are set as the x-axis, and the robot's left direction is the positive x-axis direction; the robot's forward and backward directions are set as the y-axis, and the robot's rear direction is the positive y-axis direction; the robot's vertical direction is set as the z-axis, and the robot's top direction is the positive z-axis direction. The constructed coordinate system is denoted as the target coordinate system.

[0133] Figure 5 This is a preprocessing flowchart of a specific embodiment, as shown below. Figure 5As shown, when a user enters the venue, the robot's vision module performs image recognition on the user information and obtains a set of user information, {id, x1, y1}. After obtaining multiple sets of user information, time binding is performed, i.e., a fixed frequency is preset to obtain user information sets at different times according to this fixed frequency. Subsequently, distance-based weighted coefficients and angle-based weighted coefficients are constructed. The weighted coefficient for the user closest to the robot is preset to N, and the interval of the distance-based weighted coefficients between adjacent users is L. Thus, the weighted coefficient in the second position, sorted from largest to smallest, is (NL), and so on, with the weighted coefficient in the X position, sorted from largest to smallest, being [N-(X-1)L]. The weighted coefficient for the smallest angle between the user and the robot is preset to M, and the interval of the angle-based weighted coefficients between adjacent users is K. Thus, the weighted coefficient in the second position, sorted from smallest to largest angle, is set to (MK), and so on, with the weighted coefficient in the X position, sorted from smallest to largest, being [M-(X-1)K]. This concludes the user information preprocessing process.

[0134] More specifically, when a user enters the venue, the robot's vision module sends out the location information x1 and y1 of the users around the robot based on a coordinate system with the robot's location as the farthest point. Here, x1 and y1 represent the user's horizontal position coordinates relative to the robot. The robot's vision module also sends out the ID of the users around the robot. The ID is a unique identifier for the users around the robot. The ID of the same user is fixed and is represented as a positive number, starting from 1 and increasing sequentially.

[0135] Using the user location information x1 and y1 obtained above, as well as the user's identifier id information, a set of identifier information S = {id, x1, y1} is constructed; the preset frequency of the robot collecting the above information is f, for example, the preset collection frequency is 1 Hz, that is, the number of information generated in 1 second; the preset effective time for determining the target user is T, that is, using the sampling data in the time interval [t, t+T], it is determined who the target user is at time (t+T), where t is the time when the collection starts.

[0136] For example, three individuals, A, B, and C, enter the venue sequentially at 10:01 AM and move towards the robot. After the robot's vision module detects the three users, it assigns IDs to each user: user A's ID is set to 1, user B's ID to 2, and user C's ID to 3. Based on the coordinate axes with the robot as the origin, at 10:01 AM, user A's position coordinates in the target coordinate system are (10, 10, 0), user B's position coordinates are (-20, 20, 0), and user C's position coordinates are (0, 15, 0).

[0137] Step S420: Calculate the distance-based weighting coefficient between the user and the robot.

[0138] Specifically, let the set of information about the j-th person at time i be:

[0139] S i,j ={id i ,x i,j ,y i,j}

[0140] Where, x i,j ,y i,j These represent the position information of the j-th person at time i, that is, the x-axis coordinates and y-axis coordinates of the j-th person at time i in the target coordinate system, respectively. i Let represent the user identifier of the j-th person at time i. Then, the distance between the j-th person at time i and the robot is:

[0141]

[0142] Sort the distances between all characters and robots at time i from closest to furthest, assign weighting coefficients, and record their IDs to construct a set:

[0143] C i,j ={id i ,(N-(X-1)L) i,j}

[0144] Where, x i,j ,y i,j Represent the x-axis and y-axis coordinates of the j-th person at time i in the target coordinate system, respectively, [(N-(X-1)L) i,j [] represents the weighting coefficient of the j-th person at time i, and X represents the distance between the person's ID and the robot at time i, sorted from closest to furthest. For example, the weighting coefficient of the user closest to the robot is represented as (N). i,j The second closest is (NL). i,j And so on.

[0145] For example, at 10:01 AM, User A's position coordinates in the target coordinate system are (10, 10, 0), User B's position coordinates are (-20, 20, 0), and User C's position coordinates are (0, 15, 0). Therefore, at 10:01 AM, the distance between User A and the robot is 14.14 cm, the distance between User B and the robot is 28.28 cm, and the distance between User C and the robot is 15.00 cm. The distances between User A, User B, and User C, from smallest to largest, are User A, User C, and User B. Assuming the weighting coefficient for the user closest to the robot is 30, and the interval between the weighting coefficients of adjacent users is 5, then the weighting coefficient between User A and the robot is 30, the weighting coefficient between User B and the robot is 20, and the weighting coefficient between User C and the robot is 25.

[0146] Step S430: Calculate the angle-based weighting coefficient between the user and the robot.

[0147] Specifically, let the set of information about the j-th person at time i be:

[0148] S i,j ={id i ,x i,j ,y i,j}

[0149] Where, x i,j ,y i,j Let id represent the x-axis coordinates and y-axis coordinates of the j-th person at time i in the target coordinate system. i Let represent the user identifier of the j-th person at time i; construct an initial vector starting from the origin (the robot's location coordinates) and ending at the location of the j-th person:

[0150]

[0151] set up Let θ be the angle between the target coordinate system and the unit vector (0, -1, 0) in the opposite direction of the y-axis. Then, according to the law of cosines, we can obtain:

[0152]

[0153]

[0154] The corresponding characters at time i The angles between the target coordinate system and the unit vector (0, -1, 0) in the opposite direction of the y-axis are sorted from smallest to largest, assigned weighted coefficients, and their IDs are recorded to construct a set.

[0155] D i,j ={id i ,(M-(X-1)K) i,j}

[0156] Where M is the weighted coefficient with the smallest angle between the user and the robot, K is the interval between the angle-based weighted coefficients of adjacent users, and X represents the relationship between the user ID and the corresponding [user ID] at time i. The angles between the vectors and the unit vector (0, -1, 0) in the opposite direction of the y-axis are sorted from smallest to largest, with the smallest being (M). i,j The second closest is (MK). i,j And so on.

[0157] For example, at 10:01 AM, User A's position coordinates in the target coordinate system are (10, 10, 0), User B's position coordinates are (-20, 20, 0), and User C's position coordinates are (0, 15, 0). Therefore, at 10:01 AM, the angle between User A and the unit vector (0, -1, 0) in the opposite direction of the y-axis of the robot's coordinate system is 135°, and the angle between User B and the unit vector (0, -1, 0) in the opposite direction of the y-axis of the robot's coordinate system is also 135°. The angle between the unit vector (0, -1, 0) in the opposite direction of the y-axis of the robot's coordinates (where user C is located) is 180°. Therefore, the angles of users A, B, and C, in ascending order, are user A, user B, and user C. The weighting coefficient for the user closest to the robot is preset to 30, and the interval between the distance-based weighting coefficients of adjacent users is 5. Thus, the angle-based weighting coefficient between user A and the robot is 30, the angle-based weighting coefficient between user B and the robot is 30, and the angle-based weighting coefficient between user C and the robot is 25.

[0158] Step S440: Identify the target user.

[0159] Specifically, using the sampling data from the time interval [t, t+T], we determine who is a VIP user at time t+T; we obtain all sampled data at time t+T through sampling.

[0160] Q = {S1,S2,…,S} i ,…,S L}

[0161] Among them, S i ={id i ,x i ,y iLet} represent the set of information for the j-th person at time i. Calculate the weighted coefficients of the IDs corresponding to all sampled persons at time t+T based on the distance between the person and the robot within the time interval [t, t+T]. Calculate the weighted coefficients of the IDs corresponding to all sampled persons at time t+T based on the angle between the person and the robot within the time interval [t, t+T]. Sum the weighted coefficients of the IDs corresponding to all sampled persons at time t+T based on the distance between the person and the robot and the weighted coefficients based on the angle between the person and the robot within the time interval [t, t+T]. The person with the largest sum is the target user. Specifically, when there are multiple target users, adjust the ratio of the angle-based and distance-based weighted coefficients for each user, where the ratio of the angle-based weighted coefficient is greater than the ratio of the distance-based weighted coefficient.

[0162] Figure 6 Example diagram for identifying the target user in this specific embodiment. For example... Figure 6 As shown, the robot's location coordinates are set to the origin (0,0,0). A three-dimensional coordinate system is established around the robot's location, with the left and right directions defined as the x-axis (positive x-axis direction); the forward and backward directions as the y-axis (positive y-axis direction); and the vertical direction as the z-axis (positive z-axis direction). The coordinates of characters 1, 2, 3, 4, and 5 in the target coordinate system based on the robot's coordinate axes are obtained. Weighted coefficients based on distance and angle are calculated for each character at multiple moments within a preset time period. A total weighted coefficient is obtained based on a certain ratio, and the coordinates of the target user are then determined among the characters. For example, character 1 is determined as the target user at the current moment. Here, "character" refers to the aforementioned user.

[0163] For example, at 10:01 AM, User A's position coordinates in the target coordinate system are (10,10,0), User B's position coordinates in the target coordinate system are (-20,20,0), and User C's position coordinates in the target coordinate system are (0,15,0). At this time, we can obtain that the weighting coefficient based on distance between User A and the robot is 30, and the weighting coefficient based on angle is 30; the weighting coefficient based on distance between User B and the robot is 20, and the weighting coefficient based on angle is 30; and the weighting coefficient based on distance between User C and the robot is 25, and the weighting coefficient based on angle is 25. At 10:02 AM, User A's position coordinates in the target coordinate system are (12, 16, 0), User B's position coordinates in the target coordinate system are (-10, 20, 0), and User C's position coordinates in the target coordinate system are (9, 9, 0). At this time, the weighted coefficients based on distance between User A and the robot are 20 and 30 respectively; the weighted coefficients based on distance between User B and the robot are 25 and 20 respectively; and the weighted coefficients based on distance between User C and the robot are 30 and 25 respectively.

[0164] In summary, the total weighted coefficient based on distance between User A and the robot is 50, between User B and the robot is 45, and between User C and the robot is 55. The total weighted coefficient based on angle between User A and the robot is 60, between User B and the robot is 50, and between User C and the robot is 50. Therefore, the total weighted coefficient between User A and the robot is 110, between User B and the robot is 95, and between User C and the robot is 105. Thus, User A can be identified as the target user between 10:01 AM and 10:02 AM.

[0165] It should be noted that the steps shown in the above process or in the flowcharts in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions.

[0166] This embodiment also provides a target user determination device based on robot vision, which is used to implement the above embodiments and preferred embodiments; details already described will not be repeated. The terms "module," "unit," "subunit," etc., used below refer to combinations of software and / or hardware that implement a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0167] Figure 7 This is a structural block diagram of the target user determination device based on robot vision in this embodiment, as shown below. Figure 7 As shown, the device includes: an acquisition module 10, a processing module 20, and a determination module 30.

[0168] The acquisition module 10 is used to acquire position data between multiple users and the robot within a preset time period; the robot is used to acquire the user's position information based on the robot's vision module, and the position data is determined based on the user's position information and the robot's position information.

[0169] Processing module 20 is used to determine the weighting coefficients for multiple users based on location data.

[0170] The determination module 30 is used to sort multiple users according to weighting coefficients, obtain the sorting results, and determine the target user based on the sorting results.

[0171] It should be noted that the above modules can be functional modules or program modules, and can be implemented through software or hardware. For modules implemented through hardware, the above modules can reside in the same processor; or the above modules can be located in different processors in any combination.

[0172] This embodiment also provides an electronic device, including a memory and a processor, wherein the memory stores a computer program and the processor is configured to run the computer program to perform the steps in any of the above method embodiments.

[0173] Optionally, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor and the input / output device is connected to the processor.

[0174] Optionally, in this embodiment, the processor can be configured to perform the following steps via a computer program:

[0175] Step 1: Obtain position data between multiple users and the robot within a preset time period; the robot uses its vision module to obtain the user's position information, and the position data is determined based on the user's position information and the robot's position information.

[0176] Step 2: Determine the weighting coefficients for multiple users based on location data.

[0177] Step 3: Sort the multiple users according to the weighting coefficients to obtain the sorting results, and determine the target users based on the sorting results.

[0178] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementations, and will not be repeated in this embodiment.

[0179] Furthermore, in conjunction with the target user determination method based on robot vision provided in the above embodiments, this embodiment can also provide a storage medium for implementation. The storage medium stores a computer program; when executed by a processor, the computer program implements any of the target user determination methods based on robot vision in the above embodiments.

[0180] It should be understood that the specific embodiments described herein are merely illustrative of the application and not intended to limit it. All other embodiments derived by those skilled in the art based on the embodiments provided in this application without inventive effort are within the scope of protection of this application.

[0181] Obviously, the accompanying drawings are merely some examples or embodiments of this application. Those skilled in the art can apply this application to other similar situations based on these drawings without any creative effort. Furthermore, it is understood that although the work done in this development process may be complex and lengthy, for those skilled in the art, certain design, manufacturing, or production modifications made based on the technical content disclosed in this application are merely conventional technical means and should not be considered as insufficient disclosure of this application.

[0182] The term "embodiment" in this application refers to a specific feature, structure, or characteristic described in connection with an embodiment that may be included in at least one embodiment of this application. The appearance of this phrase in various places in the specification does not necessarily imply the same embodiment, nor does it imply that it is mutually exclusive with or independent of other embodiments. It will be clearly or implicitly understood by those skilled in the art that the embodiments described in this application may be combined with other embodiments without conflict.

[0183] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of patent protection. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the appended claims.

Claims

1. A method for determining target users based on robot vision, characterized in that, The method includes: The system acquires position data between multiple users and the robot within a preset time period. The robot is used to acquire user position information based on its vision module. The position data is determined based on the user position information and the robot's position information. The user position information includes user coordinates in a coordinate system constructed with the robot's position coordinates as the origin. The weighting coefficients for multiple users are determined based on the location data. The weighting coefficients are determined by the preset weighting coefficient of the user closest to the robot, the interval of the distance-based weighting coefficients between adjacent users, the preset weighting coefficient of the user with the smallest angle between the user and the robot, and the interval of the angle-based weighting coefficients between adjacent users. The weighted coefficients corresponding to multiple users at different times within a preset time period are summed to obtain a total weighted coefficient; the multiple users are sorted according to the total weighted coefficient to obtain a sorting result; and the target user is determined according to the sorting result. When there are multiple target users, the weighting coefficients based on angle and distance are adjusted. Based on the adjusted weighting coefficients, the multiple target users are re-sorted, and new target users are determined based on the re-sorting results. Among them, the weighting coefficient based on angle is greater than the weighting coefficient based on distance.

2. The target user determination method based on robot vision according to claim 1, characterized in that, The acquisition of location data between multiple users and the robot within a preset time period includes: A preset frequency for collecting location information is established, and location information of multiple users is collected according to the preset frequency. Based on the robot's location information and the location information of multiple users, position data between multiple users and the robot is determined; the position data includes the distance between the user and the robot and the angle between the user and the robot. A user identifier for a plurality of users is determined, and an information set for the plurality of users is constructed; wherein, the user identifier is unique, and the information set includes the user identifier and the location information of the user corresponding to the user identifier.

3. The target user determination method based on robot vision according to claim 2, characterized in that, The step of determining the location data between the multiple users and the robot based on the robot's location information and the location information of the multiple users includes: Construct a coordinate system with the robot's position coordinates as the origin; Based on the robot's position coordinates and the user's coordinates, determine the position data between the user and the robot.

4. The target user determination method based on robot vision according to claim 3, characterized in that, The step of determining the position data between the user and the robot based on the robot's position coordinates and the user's coordinates includes: Based on the user's coordinates, determine the distance between the user and the robot; The angle between the user and the robot is determined based on the vector corresponding to the user's coordinates and the preset direction vector of the robot. The distance between the user and the robot and the angle between the user and the robot are determined as the positional data between the user and the robot.

5. The target user determination method based on robot vision according to claim 4, characterized in that, The step of determining the weighting coefficients corresponding to multiple users based on the location data includes: Based on the information set of multiple users, determine the distance between the multiple users and the robot at a preset time point; The distances between the multiple users and the robot are sorted to obtain the sorting results; Based on the sorting results, a first weighting coefficient is determined for each of the users.

6. The target user determination method based on robot vision according to claim 5, characterized in that, The step of determining the weighting coefficients corresponding to multiple users based on the location data further includes: Based on the information sets of the multiple users, determine the direction vectors of the multiple users; The angle between the user and the robot is determined based on the preset direction vector of the robot and the direction vector of the user; The angles of the multiple users relative to the robot are sorted to obtain a sorting result; Based on the sorting results, a second weighting coefficient is determined for each of the users; The weighting coefficients for the multiple users are determined based on the first weighting coefficient and the second weighting coefficient.

7. The target user determination method based on robot vision according to claim 5, characterized in that, When multiple target users exist, the weighting coefficients based on angle and distance are adjusted, and the target users are re-ranked according to the adjusted weighting coefficients. New target users are then determined based on the re-ranking result, including: When there are multiple target users, obtain the first weighting coefficient and the second weighting coefficient for each target user. The ratio of the first weighting coefficient and the second weighting coefficient in the weighting coefficient is preset, wherein the ratio of the second weighting coefficient is greater than the ratio of the first weighting coefficient; Calculate the weighting coefficients for multiple target users according to a preset ratio; Calculate the total weighted coefficient of the multiple target users based on their weighted coefficients; The multiple total weighted coefficients are sorted to obtain a sorting result, and the target user is re-determined based on the sorting result.

8. A target user determination device based on robot vision, characterized in that, The device includes: an acquisition module, a processing module, and a determination module; the acquisition module is used to acquire position data between multiple users and the robot within a preset time period; the robot is used to acquire user position information according to the robot's vision module, and the position data is determined based on the user position information and the robot's position information; the user position information includes user coordinates in a coordinate system constructed with the robot's position coordinates as the origin; The processing module is used to determine weighting coefficients corresponding to multiple users based on the location data; the weighting coefficients are determined by the preset weighting coefficient of the user closest to the robot, the interval of the distance-based weighting coefficients between adjacent users, the preset weighting coefficient of the user with the smallest angle between the user and the robot, and the interval of the angle-based weighting coefficients between adjacent users. The determination module is used to accumulate the weighted coefficients corresponding to multiple users at different times within a preset time to obtain a total weighted coefficient; sort the multiple users according to the total weighted coefficient to obtain a sorting result; determine the target user according to the sorting result; when there are multiple target users, adjust the ratio of the user's angle-based and distance-based weighted coefficients, re-sort the multiple target users according to the adjusted weighted coefficients, and determine a new target user according to the re-sorting result; wherein, the ratio of the angle-based weighted coefficient is greater than the ratio of the distance-based weighted coefficient.

9. An electronic device comprising a memory and a processor, characterized in that, The memory stores a computer program, and the processor is configured to run the computer program to perform the target user determination method based on robot vision as described in any one of claims 1 to 7.

Images