A robot vision direction recognition method and system for robot navigation

Abstract

The application discloses a robot visual direction recognition method and system for robot navigation, which is used in the robot body orientation recognition task. Based on the image information obtained by the camera, the classification of the orientation of the robot is learned and inferred, and the orientation information is output through the integration of multiple classification labels, so that the orientation angle of the robot body can be obtained without recognizing feature points. The method comprises the following steps: acquiring a scene image to be detected, and inputting the scene image to be detected into a robot visual direction recognition model for robot navigation; the robot visual direction recognition model for robot navigation comprises a mushroom body model and a continuous attractor network; the mushroom body model is used for view direction analysis on the scene image to be detected, and a plurality of direction activity levels are generated; and the continuous attractor network is used for merging the plurality of direction activity levels, and a robot visual direction recognition result is generated.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, and particularly relates to a robot visual direction recognition method and system for robot navigation. BACKGROUND

[0002] At present, with the rapid development of robot technology, embodied robots have been applied in many fields. In the development process of embodied intelligent robots, the visual system as a core component plays a crucial role in the environmental perception, scene understanding and robot navigation of the robot.

[0003] Robot navigation is one of the most challenging capabilities required for robots, and visual direction recognition plays a key role in robot navigation. In an unknown environment, a robot can determine its position relative to the surrounding environment features through visual direction recognition. For example, the robot can recognize the direction and distance of landmark objects (such as buildings, specific signs, etc.) in the environment, and then match them with a pre-stored map or a map constructed by a simultaneous localization and mapping (SLAM) algorithm.

[0004] Existing robot visual direction recognition methods are mostly based on feature point recognition and comparison methods to calculate the position of the camera and then the orientation of the robot, but such methods rely on local feature points and require a large number of feature point comparisons, resulting in high computational complexity. SUMMARY

[0005] The present application provides a robot visual direction recognition method and system for robot navigation to solve the technical problem of high computational complexity caused by existing robot visual direction recognition methods.

[0006] The first aspect of the present application provides a robot visual direction recognition method for robot navigation, comprising:

[0007] obtaining a scene image to be detected and inputting the scene image to be detected into a robot visual direction recognition model for robot navigation; the robot visual direction recognition model for robot navigation comprises a mushroom body model and a continuous attractor network;

[0008] performing view direction analysis on the scene image to be detected by using the mushroom body model to generate a plurality of direction activity levels;

[0009] merging a plurality of direction activity levels by using the continuous attractor network to generate a robot visual direction recognition result.

[0010] Optionally, the mushroom model includes multiple projection neurons, multiple Kenyon neurons, and multiple mushroom output neurons; the step of using the mushroom model to perform view orientation analysis on the scene image to be detected, generating multiple directional activity levels, includes:

[0011] Multiple projection neurons are used to preprocess the scene image to be detected, generating multiple normalized visual features;

[0012] The normalized visual features are input into each Kenyon neuron for feature encoding to generate multiple target Kenyon cell excitation levels.

[0013] The excitation levels of multiple target Kenyon cells are used as inputs to the output neurons of each mushroom body, resulting in the output of activity levels in multiple directions.

[0014] Optionally, the process of image preprocessing of the scene image to be detected by the projection neuron includes:

[0015] The image of the scene to be detected is dimensionality reduced to generate a dimensionality-reduced scene image;

[0016] The reduced-dimensional scene image is downsampled to generate multiple visual pixels;

[0017] Each of the aforementioned visual pixels is normalized to generate a normalized visual feature corresponding to each visual pixel.

[0018] Optionally, the step of inputting the multiple normalized visual features into each Kenyon neuron for feature encoding to generate multiple target Kenyon cell excitation levels includes:

[0019] Multiple visual codes are generated by sparsely encoding each of the normalized visual features using multiple Kenyon neurons;

[0020] Based on each of the aforementioned visual codes, the initial Kenyon cell excitation level corresponding to each of the aforementioned visual codes is determined;

[0021] The initial Kenyon cell excitation levels are sorted in descending order, and the initial Kenyon cell excitation levels with a preset number of positions are selected as the corresponding target Kenyon cell excitation levels.

[0022] Optionally, the process for processing the directional activity level specifically includes:

[0023] y i =Wx i ;

[0024] Among them, y i x represents the directional activity level corresponding to the i-th visual encoding; iis the i-th visual code; W is a connection matrix, W e R m×d , the matrix elements of W are binary elements, w ij e {0, 1}, only η 0d matrix elements take the value 1, η 0 is the sampling rate, d is the dimension of x i , and m is the number of visual codes.

[0025] Optionally, the continuous attractor network, in particular:

[0026] ;

[0027] wherein, is the corresponding rate of change; is the rate of change of the corresponding variable; is the degree of excitation of the neuron at different positions at time t, is the convergence result of the robot visual direction recognition; is the neural density, , h is the number of direction discretization; is the excitatory connection from the neuron at to the neuron at o; is the firing rate of the neuron at at time t; is the direction activity level, representing the input of the neuron at o at time t; is a constant for controlling the size of the recurrent interaction; is half the width of the excitatory connection range, typically 0.1; is the firing rate of the neuron at o at time t; is the degree of inhibition, typically 0.1; is the time constant of population dynamics, typically 1ms.

[0028] The second aspect of the present application provides a robot visual direction recognition system for robot navigation, comprising:

[0029] An acquisition module is configured to acquire a scene image to be detected and input the scene image to be detected into a robot visual direction recognition model for robot navigation; the robot visual direction recognition model for robot navigation comprises a mushroom body model and a continuous attractor network.

[0030] An adoption module is configured to adopt the mushroom body model to analyze the view direction of the scene image to be detected and generate a plurality of direction activity levels.

[0031] A merging module is configured to merge the plurality of direction activity levels through the continuous attractor network and generate a robot visual direction recognition result.

[0032] The third aspect of the present application provides a computer device, comprising a memory and a processor, the memory stores a computer program, and the computer program is executed by the processor to make the processor execute the steps of the robot visual direction recognition method for robot navigation according to any one of the above.

[0033] The fourth aspect of the present application provides a computer readable storage medium, which stores a computer program, and the computer program is executed to implement the steps of the robot visual direction recognition method for robot navigation according to any one of the above.

[0034] The fifth aspect of the present application provides a computer program product, which comprises a computer program stored on a non-transitory computer readable storage medium, and the computer program comprises program instructions, wherein when the program instructions are executed by a computer, the computer executes the steps of the robot visual direction recognition method for robot navigation according to any one of the above.

[0035] From the above technical solutions, the present application has the following advantages:

[0036] The above technical solution of the present application provides a robot visual direction recognition method for robot navigation. First, a to-be-detected scene image is obtained, and the to-be-detected scene image is input into a robot visual direction recognition model for robot navigation. The robot visual direction recognition model for robot navigation comprises a mushroom body model and a continuous attractor network. Then, the mushroom body model is used to analyze the view direction of the to-be-detected scene image to generate multiple direction activity levels. Finally, the continuous attractor network is used to merge the multiple direction activity levels to generate a robot visual direction recognition result. Based on the above solution, the mushroom body model and the continuous attractor network are used to process the obtained to-be-detected scene image, and the process of outputting the robot visual direction recognition result. In the robot body orientation recognition task, the present application can learn and infer the classification of the orientation of the robot itself based on the image information obtained by the camera of the robot itself, and output the orientation information through the integration of multiple classification labels, so as to obtain the orientation angle of the robot body without recognizing feature points, thereby reducing the computational complexity. BRIEF DESCRIPTION OF DRAWINGS

[0037] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed in the embodiments or prior art description will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor.

[0038] Figure 1A step flow chart of a robot visual direction recognition method for robot navigation is provided for the embodiment one of the present application.

[0039] Figure 2 A structural framework diagram of a robot visual direction recognition model processing for a picture collected by a robot at 0° is provided for the embodiment one of the present application.

[0040] Figure 3 A structural block diagram of a robot visual direction recognition system for robot navigation is provided for the embodiment two of the present application. DETAILED DESCRIPTION

[0041] The embodiment of the present application provides a robot visual direction recognition method and system for robot navigation, and solves the technical problem of high calculation complexity caused by the existing robot visual direction recognition method.

[0042] In order to make the technical scheme of the present application more obvious and easy to understand, the technical scheme in the embodiment of the present application will be described clearly and completely in conjunction with the drawings in the embodiment of the present application. Obviously, the following described embodiments are only a part of the embodiments of the present application, but not all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the protection scope of the present application.

[0043] Please refer to Figure 1 , Figure 1 A step flow chart of a robot visual direction recognition method for robot navigation is provided for the embodiment one of the present application.

[0044] The robot visual direction recognition method for robot navigation provided by the present application comprises the following steps.

[0045] In step 101, a to-be-detected scene image is acquired, and the to-be-detected scene image is input to a robot visual direction recognition model for robot navigation; the robot visual direction recognition model for robot navigation comprises a mushroom body model and a continuous attractor network.

[0046] It should be noted that the robot visual direction recognition model for robot navigation provided by the present application is composed of a mushroom body model and a continuous attractor network, wherein the mushroom body model is composed of a plurality of projection neurons (PNs), a plurality of Kenyon neurons (KCs), and a plurality of mushroom output neurons (MBONs), and the synaptic plasticity from the Kenyon cell (Kenyon neuron) to the mushroom output neuron (MBON) is controlled by a biologically reasonable learning rule; the continuous attractor network (CANN) is a super-theoretical model, which is proposed before the connection group and function of the central complex are revealed, and is confirmed to be correct after the revelation. It can form a ring-shaped attractor, and neurons encode information in a group coding manner (Wu et al., 2016). When there is an external signal input, the network will generate a wave packet activity in the form of a Gaussian shape. The dynamics of CANN can act as a filter in time and space, adjusting the jitter of the input signal. In head direction coding, the head direction neurons in the brain are connected by mutual feedback, forming a one-dimensional CANN (Zhang, 1996). The collective activity of neurons in the network dynamically creates a Gaussian wave packet, and the peak position of the wave packet encodes the head direction. As the animal's head rotates, the wave packet in the network will rotate due to synaptic plasticity, thus real-time retaining the information about the head direction in the brain and maintaining our spatial orientation.

[0047] In the embodiment, a scene image to be detected is acquired, and the scene image to be detected is input into the robot visual direction recognition model for robot navigation; the robot visual direction recognition model for robot navigation includes a mushroom body model and a continuous attractor network.

[0048] Step 102, view direction analysis of the scene image to be detected is performed by using the mushroom body model, and a plurality of direction activity levels are generated.

[0049] The mushroom body model includes a plurality of projection neurons, a plurality of Kenyon neurons, and a plurality of mushroom output neurons.

[0050] Step 102 can include the following sub-steps S21-S23:

[0051] Step S21, a plurality of projection neurons are respectively used to perform image preprocessing on the scene image to be detected, and a plurality of normalized visual features are generated;

[0052] Optionally, the process of image preprocessing on the scene image to be detected by the projection neuron includes the following sub-steps S1-S3:

[0053] Step S1, dimension reduction is performed on the scene image to be detected, and a dimension-reduced scene image is generated;

[0054] Step S2, down-sampling the dimension-reduced scene image to generate a plurality of visual pixel points;

[0055] Step S3, normalizing each visual pixel point to generate a normalized visual feature corresponding to each visual pixel point.

[0056] It should be noted that the scene image to be detected has two dimensions. The present application first flattens the image into one dimension. The actual preprocessing of visual signals by insects can be complex. According to the previous model, the architecture of the mushroom body model can be processed and learned on the image without complex feature extraction, only down-sampling and normalization (Ardin et al., 2016; Dasgupta et al., 2017). Each projection neuron performs image preprocessing operations on the scene image to be detected, so that each projection neuron outputs a plurality of normalized visual features.

[0057] Step S22, inputting the plurality of normalized visual features into each Kenyon neuron for feature encoding to generate a plurality of target Kenyon cell excitation levels;

[0058] Further, step S22 can include the following sub-steps S221-S223:

[0059] Step S221, sparse coding each normalized visual feature by the plurality of Kenyon neurons to generate a plurality of visual encodings;

[0060] Step S222, determining an initial Kenyon cell excitation level corresponding to each visual encoding based on the visual encoding;

[0061] Step S223, sorting the plurality of initial Kenyon cell excitation levels in descending order, and selecting the initial Kenyon cell excitation levels of the first preset number of positions as the corresponding target Kenyon cell excitation levels.

[0062] It should be noted that the Kenyon neuron calculates the sparse representation of the normalized visual feature, which is used to model the connection from the projection neuron (PN) to the Kenyon cell (KC). The mapping from the PN to the KC can be regarded as a sparse, binary, randomly generated connection matrix W.

[0063] Specifically, each Kenyon neuron sparse encodes the plurality of input normalized visual features, so that each Kenyon neuron outputs a plurality of visual encodings, and determines an initial Kenyon cell excitation level corresponding to each visual encoding based on the visual encoding.

[0064] Further, the initial Kenyon cell excitation degrees are sorted in descending order, and the top preset number of initial Kenyon cell excitation degrees are selected as the corresponding target Kenyon cell excitation degrees; wherein, the preset number can be set as needed, and the application is not limited thereto, for example, if the preset number is set as 20, then after the initial Kenyon cell excitation degrees are sorted in descending order, the top 20 initial Kenyon cell excitation degrees are selected as the corresponding target Kenyon cell excitation degrees.

[0065] In step S23, the multiple target Kenyon cell excitation degrees are input into each mushroom body output neuron respectively, and multiple direction activity levels are output.

[0066] It should be noted that each mushroom body output neuron processes the input multiple target Kenyon cell excitation degrees, so that each mushroom body output neuron outputs multiple direction activity levels, wherein the output (direction activity level) of the mushroom body model is a multi-class label likelihood, which is input into a simplified central complex (CX) model simulated by a continuous attractor network; wherein the processing process of the direction activity level is as follows:

[0067] y i =Wx i ;

[0068] wherein y i is the direction activity level corresponding to the i th visual code; x i is the i th visual code, which is the visual code corresponding to the target Kenyon cell excitation degree; W is a connection matrix, W∈R m×d , the matrix elements of W are binary elements, w ij ∈0,1, only η0d matrix elements take the value 1, η0 is the sampling rate, and d is the dimension of x i .

[0069] It is worth mentioning that unlike typical multi-class classification datasets that have no correlation between labels, the dataset in the application shows correlation between labels, allowing interpolation of the output of “MBONs” to obtain finer direction resolution. In insects, multiple MBONs are connected to the fan-shaped body of the central complex (CX), i.e. multiple MBONs are connected to the continuous attractor network. Since CANN (continuous attractor network) has been proven to be a simplified model of CX, the application constructs a robot visual direction recognition model for robot navigation by modifying CANN to receive multiple outputs from “MBONs”.

[0070] In this embodiment, the mushroom body model is used to analyze the view direction of the scene image to be detected, and multiple direction activity levels are generated.

[0071] Step 103: Merge the activity levels of multiple directions through a continuous attractor network to generate robot vision orientation recognition results.

[0072] It should be noted that a continuous attractor network describes a loop attractor through multiple interconnected neurons. Each neuron is assigned a direction, stimulating distant neurons and inhibiting nearby neurons. Specifically, the continuous attractor network is as follows:

[0073] ;

[0074] in, for The corresponding rate of change; To represent the excitation level of neurons at different locations at time t, The convergence result is the robot's visual orientation recognition result. For nerve density, h is the number of directional discretizations; From Excitatory connections from neuron at position O to neuron at position O; For time t The firing rate of neurons; , where represents the directional activity level, indicating the input of the neuron at point o at time t; This is a constant used to control the magnitude of the loop interactions; It is half the width of the excitatory connectivity range, with a typical value of 0.1; Let be the firing rate of the neuron at point o at time t; The degree of inhibition is typically 0.1; This is the time constant for population dynamics, typically 1 ms.

[0075] It is worth mentioning that the contribution of inhibiting connections is through The division normalization in the model is indirectly achieved, and the output of the simplified mushroom model is achieved through... The data is fed into CANN (Continuous Attractor Network), where o corresponds to the label "MBONs". Therefore, this invention can add more neurons to CANN to interpolate and obtain a finer resolution output.

[0076] Furthermore, As the excitation level of different MBON neurons at time t, after passing through a continuous attractor network, the excitation level of neurons at the corresponding angle of the robot is the highest, thus determining the robot's orientation.

[0077] Further, based on the simplified mushroom model for angle retrieval and the simplified central complex model with a continuous attractor neural network (CANN) proposed by the present application, a robot visual direction recognition model for robot navigation is constructed. The former associates the view of an object with the direction angle. The latter combines multiple outputs of the former, thereby providing fine direction information (robot visual direction recognition result) for visual positioning of the robot. The test of the present application on robot navigation proves the effectiveness of FlyOrien in the dataset and practical application, and shows that it is superior to the traditional artificial neural network in efficiency.

[0078] Specifically, in order to simulate the behavior of animals finding direction, two-part experiments were conducted on a quadruped robot for verification. In Experiment 1, the robot found a familiar object or landmark in the environment and made a 360° angle judgment around the landmark. In the second experiment, in an empty scene without suitable objects or landmarks to surround, the robot judged its own angle by rotating itself. A total of 360 images were collected for each sample, each image being 128x128 pixels in size. Compared with the test results of the redesigned dataset, the angle interval of the test set in this experiment changed from 10° to 1°, which is more dense and more consistent with the randomness of the robot's angle and position in the real scene.

[0079] The common control method relies on inertial odometry, and the goal of the present application is to control the robot along a circular trajectory, facing the center of the circle. The typical radius of the circle is 0.5m, and the typical speed is 1-1.5deg / s. The on-board control system is remotely controlled by our server, communicating through the existing Unitree protocol implementation.

[0080] In the robot visual positioning experiment on objects, the present application realizes sampling from different angles when the robot rotates 360° around the object through robot motion control and image sampling algorithm. The robot rotates around the object, taking a photo every degree with the head camera, a total of 360 photos. The binocular fisheye lens on the robot's head has a 180° field of view. Through the official camera calibration algorithm built-in the robot, the corrected photos are transmitted in real time during the sampling process. The image size of the left and right eyes is 800x928. We will use the image from the left eye in the subsequent experiment and compress it to 128x128. Every 10 degrees, a set of sample photos is selected as the training dataset, and the rest is used as the test set, with the nearest 10° as the label for accuracy testing. The cup, foam box and plant were sampled and tested to verify the accuracy. The unclipped dataset was also tested. According to the results, the similarity predicted by the network has an accuracy of more than 96% in the top 5 angles and more than 80% in the top 2 angles.

[0081] In the robot visual localization experiment, the robot can only rotate in a circle by setting the rotation speed. The robot is sampled in three different scenes: lab 1, lab 2 and corridor. The sampling method is the same as the previous experiment. According to the results, the similarity of the predicted angles is more than 96% accurate in the top 5 angles, and more than 80% accurate in the top 2 angles. On the dense test set with an interval of 1°, the performance remains relatively stable.

[0082] By way of example, reference is made to Figure 2 When the robot collects the scene image at 0°, the robot visual direction recognition result output by the mushroom model and the continuous attractor network indicates that the robot is currently facing 0°.

[0083] In this embodiment, the continuous attractor network is used to combine multiple direction activity levels to generate a robot visual direction recognition result.

[0084] As a comparison of technical effects, reference can be made to the prior art. In a natural environment, many cues can indicate direction, such as the direction of the sun, the shift of sunlight, the direction of the wind, and landmark buildings. Animals can use them for direction and navigation (Heinze, 2017). Most of these cues are perceived through vision. Even a simple insect can use visual memory to remember the way home, and after walking along the route once, it determines the route home by finding a familiar direction (Ardin et al., 2016). When remembering directions, their lightweight neural circuits are shorter than typical artificial neural networks (ANN). Therefore, by studying the structure and learning rules of these circuits and retrieving direction memories from visual signals, the robot visual localization can be achieved.

[0085] The intention of the present application to retrieve direction is related to object pose estimation, but differs in many aspects. A typical object pose estimation process constructs a planar map of an object with planar feature references, or uses a three-dimensional model for reference, such as a CAD model constructed for the shape of the object (Van de Ven et al., 2022). These references are accurate and have perfect memories. Through these references, the pose of the object in the image can be estimated. These types of models have a wide range of applications in face pose detection and industrial robot part picking (Fan et al., 2022). In a natural environment, objects with planar maps are not common, and animals do not have pre-defined perfect memories for reference. For animals, the relative direction of an object is more important than the pose of the object for navigation on the ground. Therefore, the robot visual direction recognition model for robot navigation proposed by the present application focuses on relative direction rather than pose estimation.

[0086] Biological experiments and computational neuroscience models suggest that MB plays an important role in navigation tasks (Webb and Wystrach, 2016) such as visual homing. When a desert ant leaves the nest to forage, its pheromones evaporate in high temperatures. Therefore, it has to rely on skylight and landmarks for navigation Ardin et al. (2016). Ardin et al. (2016) proposed a MB model in which ants can observe the surrounding environment, and MBONs can indicate the unfamiliarity of the head direction according to the visual input. According to the model, ants can learn the familiarity of the scene direction on the route away from the nest, and then go home by searching the most familiar oncoming direction.

[0087] Another neural circuit that has been proven to be crucial in navigation tasks is the central complex (CX) (Honkanen et al. 52, 2019). Neurons in CX form a ring attractor to encode information about head direction and homeward direction in a population coding manner. It can combine multiple directional cues to find a more accurate direction than a single cue (Heinze, 2017). Neurons representing different directions integrate the moving speed, in this way, CX can calculate and memorize the direction and distance from the nest (Stone et al., 2017; Franconville et al., 2018). The computational neuroscience model predicts neuronal activity that matches the ring attractor, which is consistent with biological observations.

[0088] Based on the above foundation, the present application combines a simplified mushroom body model for angle retrieval and a simplified central complex model with a continuous attractor neural network (CANN) (Wu et al., 2016) to obtain a robot visual direction recognition model for robot navigation. The former associates the view of an object with the direction angle. The latter combines multiple outputs of the former to provide fine direction information for visual positioning of the robot. Among them, visual direction recognition helps navigation, especially in the absence of a reliable magnetic compass or GPS. Based on the neural mechanism of the brain, the present application combines a simplified mushroom body model for angle retrieval and a simplified central complex model with a continuous attractor neural network (CANN) (Wu et al., 2016) to obtain a robot visual direction recognition model for robot navigation, which provides multi-cue sensor integration, supports interpolation and finer direction representation through the continuous attractor neural network.

[0089] In the embodiment of the present application, the present application provides a robot visual direction recognition method for robot navigation. First, a to-be-detected scene image is acquired, and the to-be-detected scene image is input into a robot visual direction recognition model for robot navigation. The robot visual direction recognition model for robot navigation includes a mushroom body model and a continuous attractor network. Then, the mushroom body model is used to analyze the view direction of the to-be-detected scene image to generate multiple direction activity levels. Finally, the continuous attractor network is used to merge the multiple direction activity levels to generate a robot visual direction recognition result. Based on the above scheme, the acquired to-be-detected scene image is processed by using the mushroom body model and the continuous attractor network, and the process of outputting the robot visual direction recognition result. The mushroom body model can learn the familiarity of the scene direction, and the continuous attractor network can merge multiple direction clues to form more comprehensive and accurate direction information. At the same time, the present application can learn and infer the classification of the orientation of the robot itself based on the image information obtained by the camera of the robot itself in the robot body orientation recognition task, and output the orientation information by integrating multiple classification labels, so as to obtain the orientation angle of the robot body without recognizing feature points, thereby reducing the computational complexity.

[0090] In addition, the existing robot visual direction recognition method is mostly based on the feature point recognition and comparison method, calculates the position of the camera, and then calculates the orientation of the robot. This method depends on local feature points, needs to compare a large number of feature points, has high computational complexity, and has low reliability in the case of insufficient texture. Moreover, the global features of the image cannot be directly used. However, the present application does not need to compare feature points, the number of feature points does not affect the computational complexity, does not depend on texture, and can directly use the global features of the image.

[0091] Please refer to Figure 3 , Figure 3 The structure block diagram of a robot visual direction recognition system for robot navigation according to the second embodiment of the present application is provided.

[0092] The robot visual direction recognition system for robot navigation provided by the present application comprises:

[0093] The acquisition module 301 is configured to acquire a to-be-detected scene image and input the to-be-detected scene image into a robot visual direction recognition model for robot navigation. The robot visual direction recognition model for robot navigation includes a mushroom body model and a continuous attractor network.

[0094] The using module 302 is configured to use the mushroom body model to analyze the view direction of the to-be-detected scene image to generate multiple direction activity levels.

[0095] The merging module 303 is configured to merge the multiple direction activity levels by using the continuous attractor network to generate a robot visual direction recognition result.

[0096] Further, the mushroom body model comprises multiple projection neurons, multiple Kenyon cells, and multiple mushroom body output neurons.

[0097] The first sub-module is configured to use the multiple projection neurons to respectively perform image preprocessing on the to-be-detected scene image to generate multiple normalized visual features.

[0098] The second sub-module is configured to input the multiple normalized visual features into the respective Kenyon neurons for feature encoding to generate multiple target Kenyon cell excitation levels.

[0099] The third sub-module is configured to input the multiple target Kenyon cell excitation levels into the respective mushroom body output neurons to output multiple direction activity levels.

[0100] Further, the first sub-module is specifically configured to:

[0101] perform dimensionality reduction on the to-be-detected scene image to generate a dimensionality-reduced scene image;

[0102] perform down-sampling on the dimensionality-reduced scene image to generate multiple visual pixel points;

[0103] respectively normalize the visual pixel points to generate normalized visual features corresponding to the visual pixel points.

[0104] Further, the second sub-module is specifically configured to:

[0105] perform sparse coding on the respective normalized visual features by using the multiple Kenyon neurons to generate multiple visual encodings;

[0106] determine initial Kenyon cell excitation levels corresponding to the visual encodings based on the visual encodings;

[0107] perform descending sorting on the multiple initial Kenyon cell excitation levels, and select the initial Kenyon cell excitation levels in the first preset number of positions as the corresponding target Kenyon cell excitation levels.

[0108] Further, the processing process of the direction activity level is specifically as follows:

[0109] y i =Wx i ;

[0110] wherein y i is a direction activity level corresponding to the i-th visual encoding; x i is the i-th visual encoding; and W is a connection matrix, W ∈ Rm×d The matrix elements of W are binary elements, w ij ∈0,1, only η0d matrix elements have a value of 1, where η0 is the sampling rate and d is the value of x. i The dimension of ; m is the number of visual codes.

[0111] Furthermore, the continuous attractor subnetwork is specifically as follows:

[0112] ;

[0113] in, for The corresponding rate of change; To represent the excitation level of neurons at different locations at time t, The convergence result is the robot's visual orientation recognition result. For nerve density, h is the number of directional discretizations; From Excitatory connections from neuron at position O to neuron at position O; For time t The firing rate of neurons; , where represents the directional activity level, indicating the input of the neuron at point o at time t; This is a constant used to control the magnitude of the loop interactions; It is half the width of the excitatory connectivity range, with a typical value of 0.1; Let be the firing rate of the neuron at point o at time t; The degree of inhibition is typically 0.1; This is the time constant for population dynamics, typically 1 ms.

[0114] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, modules, and sub-modules described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0115] This invention also provides a computer device, including a memory and a processor, wherein the memory stores a computer program; when the computer program is executed by the processor, the processor performs the steps of the robot vision orientation recognition method for robot navigation as described in Embodiment 1 above.

[0116] This invention also provides a computer-readable storage medium storing a computer program / instructions thereon, which, when executed by a processor, implements the steps of the robot vision orientation recognition method for robot navigation as described in Embodiment 1 above.

[0117] The embodiment of the present application further provides a computer program product comprising computer programs / instructions, which, when executed by a processor, implement the steps of the robot visual direction recognition method for robot navigation of the above embodiment one.

[0118] In several embodiments provided in the present application, it should be understood that the disclosed system and method can be implemented in other manners. For example, the above described unit embodiments are merely schematic, and the unit division is merely a logical function division, and there can be another division manner for the actual implementation, for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections between different units, or the among different units, can be indirect couplings or communication connections through some interfaces, devices or units, and can be in electrical, mechanical or other forms.

[0119] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they can be located in one place, or can be distributed on multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment scheme.

[0120] The above described, the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that the technical solutions recorded in the foregoing embodiments can still be modified, or some technical features can be replaced by equivalents; and these modifications or replacements do not make the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A robot visual direction recognition method for robot navigation, characterized by, The method comprises the following steps: obtaining a scene image to be detected and inputting the scene image to be detected into a robot visual direction recognition model for robot navigation; the robot visual direction recognition model for robot navigation comprises a mushroom body model and a continuous attractor network; the mushroom body model comprises a plurality of projection neurons, a plurality of Kenyon neurons and a plurality of mushroom body output neurons; using the mushroom body model to perform view direction analysis on the scene image to be detected to generate a plurality of direction activity levels, which comprises: using a plurality of projection neurons to respectively perform image preprocessing on the scene image to be detected to generate a plurality of normalized visual features; inputting the plurality of normalized visual features into each Kenyon neuron respectively to perform feature coding to generate a plurality of target Kenyon cell excitation levels; inputting the plurality of target Kenyon cell excitation levels into each mushroom body output neuron respectively to output a plurality of direction activity levels; merging the plurality of direction activity levels through the continuous attractor network to generate a robot visual direction recognition result; wherein the continuous attractor network describes a ring attractor through a plurality of interconnected neurons, each neuron is assigned a direction, stimulates distant neurons and inhibits nearby neurons, and after passing through the continuous attractor network, the excitation level of the neuron corresponding to the angle of the robot is the highest, so as to determine the direction of the robot.

2. The robot vision direction recognition method for robot navigation according to claim 1, characterized in that, The process of image preprocessing on the scene image to be detected by the projection neuron comprises: dimensionality reduction is performed on the scene image to be detected to generate a dimensionality reduced scene image; down-sampling is performed on the dimensionality reduced scene image to generate a plurality of visual pixel points; each visual pixel point is normalized to generate a normalized visual feature corresponding to each visual pixel point.

3. The method for robot vision orientation recognition for robot navigation according to claim 1, wherein, The process of inputting the plurality of normalized visual features into each Kenyon neuron respectively to perform feature coding to generate a plurality of target Kenyon cell excitation levels comprises: sparse coding is performed on each normalized visual feature through a plurality of Kenyon neurons to generate a plurality of visual codes; based on each visual code, an initial Kenyon cell excitation level corresponding to each visual code is determined; the plurality of initial Kenyon cell excitation levels are sorted in descending order, and the initial Kenyon cell excitation levels in the first preset number of positions are selected as the corresponding target Kenyon cell excitation levels.

4. The method for robot vision orientation recognition for robot navigation according to claim 1, wherein, The processing process of the direction activity level is specifically: y i = Wx i ; where y i is the direction activity level corresponding to the i-th visual code; x i is the i-th visual code; W is a connection matrix, W ∈ R m×d , the matrix elements of W are binary elements, w ij ∈ 0, 1, only η0d matrix elements take the value 1, η0 is the sampling rate, d is the dimension of x i ; m is the number of visual codes.

5. The method for robot vision orientation recognition for robot navigation according to claim 1, wherein, the continuous attractor network is specifically: ； wherein, is the corresponding rate of change; is the level of excitation of the neuron at location o at time t, after convergence is the result of the robot's visual direction recognition; is the neural density, h is the number of direction discretization; is the excitatory connection from the neuron at location o at time t to the neuron at location o at time t; is the excitatory connection from the neuron at location o at time t to the neuron at location o at time t; is the firing rate of the neuron at location o at time t; is the firing rate of the neuron at location o at time t; is the direction activity level, representing the input to the neuron at location o at time t; is a constant used to control the size of the recurrent interaction; is half the width of the excitatory connection range, with a typical value of 0.1; is the firing rate of the neuron at location o at time t; is the degree of inhibition, with a typical value of 0.1; is the time constant of the population dynamics, with a typical value of 1 ms; is the firing rate of the neuron at location o at time t; is the level of excitation of the neuron at location o at time t.

6. A robot visual direction recognition system for robot navigation, applied to the robot visual direction recognition method for robot navigation according to claim 1, characterized in that, comprises: an acquisition module, configured to acquire a scene image to be detected and input the scene image to be detected into a robot visual direction recognition model for robot navigation; the robot visual direction recognition model for robot navigation comprises a mushroom body model and a continuous attractor network; an adoption module, configured to use the mushroom body model to perform view direction analysis on the scene image to be detected to generate a plurality of direction activity levels; a merging module, configured to merge the plurality of direction activity levels through the continuous attractor network to generate a robot visual direction recognition result.

7. A computer device, comprising: A computer program product comprising a computer program stored in a non-transitory computer readable storage medium, the computer program comprising program instructions, wherein the program instructions, when executed by a computer, cause the computer to perform the steps of the robot visual direction recognition method for robot navigation according to any one of claims 1-5.

8. A computer readable storage medium having stored thereon a computer program, characterized in that, The computer program product comprises a computer program stored in a non-transitory computer readable storage medium, the computer program comprising program instructions, wherein the program instructions, when executed by a computer, cause the computer to perform the steps of the robot visual direction recognition method for robot navigation according to any one of claims 1-5.

9. A computer program product, characterised in that, The computer program product comprises a computer program stored in a non-transitory computer readable storage medium, the computer program comprising program instructions, wherein the program instructions, when executed by a computer, cause the computer to perform the steps of the robot visual direction recognition method for robot navigation according to any one of claims 1-5.

Images