Robot visual direction identification method and system for robot navigation

Abstract

The invention discloses a robot visual direction identification method and system for robot navigation, which are used for learning and deducing own orientation classification based on image information obtained by an own camera in a robot body orientation identification task, and outputting orientation information through integration of a plurality of classification labels. Therefore, the orientation angle of the robot body can be obtained without identifying the feature points. The method comprises the following steps: acquiring a to-be-detected scene image, and inputting the to-be-detected scene image into a robot visual direction recognition model for robot navigation; the robot visual direction recognition model for robot navigation comprises a mushroom-shaped body model and a continuous attractor network. Performing view direction analysis on the to-be-detected scene image by adopting a mushroom-shaped body model to generate active levels in multiple directions; and combining the active levels in the multiple directions through a continuous attractor network to generate a robot visual direction identification result.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a robot visual direction recognition method and system for robot navigation. Background Art

[0002] At present, with the rapid development of robotics technology, embodied robots have been applied in many fields. In the development of embodied intelligent robots, the visual system, as its core component, plays a vital role in the robot's environmental perception, scene understanding and robot navigation.

[0003] Robot navigation is one of the most challenging capabilities required of robots, and visual direction recognition plays a key role in robot navigation. In an unknown environment, a robot can use visual direction recognition to determine its position relative to the features of the surrounding environment. For example, a robot can identify the direction and distance of landmark objects in the environment (such as buildings, specific signs, etc.), and then match them with a pre-stored map or a map built by a simultaneous localization and mapping (SLAM) algorithm.

[0004] Most of the existing robot visual direction recognition methods are based on feature point recognition and comparison methods to infer the position of the camera and then calculate the robot's orientation. However, this method relies on local feature points and requires a large number of feature point comparisons, resulting in high computational complexity. Summary of the invention

[0005] The present invention provides a robot vision direction recognition method and system for robot navigation, which are used to solve the technical problem of high computational complexity caused by the existing robot vision direction recognition method.

[0006] A first aspect of the present invention provides a robot visual direction recognition method for robot navigation, comprising:

[0007] Acquire a scene image to be detected, and input the scene image to be detected into a robot visual direction recognition model for robot navigation; the robot visual direction recognition model for robot navigation includes a mushroom body model and a continuous attractor network;

[0008] Using the mushroom-shaped body model to perform view direction analysis on the scene image to be detected, and generating multiple directional activity levels;

[0009] The plurality of directional activity levels are merged through the continuous attractor network to generate a robot visual direction recognition result.

[0010] Optionally, the mushroom body model includes a plurality of projection neurons, a plurality of Kenyon neurons, and a plurality of mushroom body output neurons; the mushroom body model is used to perform view direction analysis on the scene image to be detected to generate a plurality of directional activity levels, including:

[0011] Using a plurality of projection neurons to perform image preprocessing on the scene image to be detected, respectively, to generate a plurality of normalized visual features;

[0012] Inputting the plurality of normalized visual features into respective Kenyon neurons for feature encoding to generate a plurality of target Kenyon cell excitation levels;

[0013] The excitation levels of the target Kenyon cells are respectively used as inputs of the mushroom body output neurons to output multiple directional activity levels.

[0014] Optionally, the process of the projection neuron performing image preprocessing on the scene image to be detected includes:

[0015] Performing dimensionality reduction on the scene image to be detected to generate a dimensionality-reduced scene image;

[0016] Down-sampling the dimension-reduced scene image to generate a plurality of visual pixel points;

[0017] Each of the visual pixel points is normalized to generate a normalized visual feature corresponding to each of the visual pixel points.

[0018] Optionally, the step of inputting the plurality of normalized visual features into respective Kenyon neurons for feature encoding to generate a plurality of target Kenyon cell excitation levels comprises:

[0019] Sparsely encoding each of the normalized visual features through a plurality of Kenyon neurons to generate a plurality of visual codes;

[0020] Based on each of the visual codes, determining the initial Kenyon cell excitation level corresponding to each of the visual codes;

[0021] The multiple initial Kenyon cell excitation levels are sorted in descending order, and the initial Kenyon cell excitation levels of the first preset number of digits are selected as the corresponding target Kenyon cell excitation levels.

[0022] Optionally, the processing process of the directional activity level is specifically as follows:

[0023] y i =Wx i ;

[0024] Among them, y i is the directional activity level corresponding to the i-th visual encoding; x iis the i-th visual encoding; W is the connection matrix, W∈R m×d , the matrix elements of W are binary elements, w ij ∈0,1, only η0d matrix elements take the value 1, η0 is the sampling rate, d is x i The dimension of ; m is the number of visual codes.

[0025] Optionally, the continuous attractor network is specifically:

[0026] ;

[0027] in, for The corresponding rate of change; is the excitation level of neurons at different positions at time t, After convergence, it is the robot visual direction recognition result; is the nerve density, , h is the number of direction discretization; For The excitatory connection from the neuron at position A to the neuron at position O; is the The firing rate of neurons at is the directional activity level, representing the input of the neuron at time t at o; is a constant used to control the magnitude of the cyclic interaction; is the half width of the excitatory connection range, with a typical value of 0.1; is the firing rate of the neuron at time t at o; is the degree of inhibition, with a typical value of 0.1; is the time constant of population dynamics, with a typical value of 1ms.

[0028] A second aspect of the present invention provides a robot visual direction recognition system for robot navigation, comprising:

[0029] An acquisition module, used for acquiring a scene image to be detected, and inputting the scene image to be detected into a robot visual direction recognition model for robot navigation; the robot visual direction recognition model for robot navigation includes a mushroom body model and a continuous attractor network;

[0030] An adopting module, configured to use the mushroom-shaped body model to perform view direction analysis on the scene image to be detected, and generate multiple directional activity levels;

[0031] A merging module is used to merge the multiple directional activity levels through the continuous attractor network to generate a robot visual direction recognition result.

[0032] A third aspect of the present invention provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the robot visual direction recognition method for robot navigation as described in any one of the above items.

[0033] A fourth aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed, implements the steps of the robot visual direction recognition method for robot navigation as described in any one of the above items.

[0034] A fifth aspect of the present invention provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium, and the computer program includes program instructions, wherein when the program instructions are executed by a computer, the computer executes the steps of the robot visual direction recognition method for robot navigation as described in any one of the above items.

[0035] It can be seen from the above technical solutions that the present invention has the following advantages:

[0036] The above technical scheme of the present invention provides a robot visual direction recognition method for robot navigation. First, an image of a scene to be detected is obtained, and the image of the scene to be detected is input into a robot visual direction recognition model for robot navigation; the robot visual direction recognition model for robot navigation includes a mushroom model and a continuous attractor network; then, the mushroom model is used to perform view direction analysis on the image of the scene to be detected to generate multiple direction activity levels; finally, the multiple direction activity levels are merged through a continuous attractor network to generate a robot visual direction recognition result; based on the above scheme, the mushroom model and the continuous attractor network are used to process the acquired image of the scene to be detected, and the robot visual direction recognition result is output. The present invention can learn and infer the classification of its own orientation based on the image information obtained by the robot's own camera in the robot body orientation recognition task, and output the orientation information through the integration of multiple classification labels, so as to obtain the robot body orientation angle without identifying feature points, thereby reducing the computational complexity. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0038] Figure 1A flowchart of a robot visual direction recognition method for robot navigation provided in Embodiment 1 of the present invention;

[0039] Figure 2 A structural framework diagram of a picture collected by a robot at 0° and processed by a robot visual direction recognition model for robot navigation provided in the first embodiment of the present invention;

[0040] Figure 3 A structural block diagram of a robot visual direction recognition system for robot navigation is provided in accordance with an embodiment of the present invention. DETAILED DESCRIPTION

[0041] The embodiments of the present invention provide a robot vision direction recognition method and system for robot navigation, which are used to solve the technical problem of high computational complexity caused by the existing robot vision direction recognition method.

[0042] In order to make the purpose, features and advantages of the present invention more obvious and easy to understand, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described below are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0043] See also Figure 1 , Figure 1 A flowchart of the steps of a robot visual direction recognition method for robot navigation provided in Embodiment 1 of the present invention.

[0044] The present invention provides a robot visual direction recognition method for robot navigation, comprising:

[0045] Step 101: Acquire a scene image to be detected, and input the scene image to be detected into a robot visual direction recognition model for robot navigation; the robot visual direction recognition model for robot navigation includes a mushroom model and a continuous attractor network.

[0046] It should be noted that the robot visual direction recognition model for robot navigation proposed in the present invention is composed of a mushroom body model and a continuous attractor network, wherein the mushroom body model is composed of multiple projection neurons (PNs), multiple Kenyon neurons (KCs), and multiple mushroom body output neurons (MBONs), and the synaptic plasticity from Kenyon cells (Kenyon neurons) to mushroom body output neurons (MBONs) is controlled by biologically reasonable learning rules; the continuous attractor network (CANN) is a super-theoretical model, which was proposed before the central complex connection group and function were revealed, and was confirmed to be correct after the revelation. It can form a ring attractor, and neurons encode information in a group coding manner (Wu et al., 2016). When there is an external signal input, the network will generate a wave packet activity with a Gaussian shape. The dynamics of CANN can act as a filter in time and space to adjust the jitter of the input signal. In head direction coding, the head direction neurons in the brain are connected to each other through mutual feedback to form a one-dimensional CANN (Zhang, 1996). The collective activity of neurons in the network dynamically creates a Gaussian wave packet, the peak position of which encodes the direction of the head. As the animal's head rotates, the wave packets in the network rotate with it due to synaptic plasticity, thus retaining information about the direction of the head in the brain in real time and maintaining our sense of spatial orientation.

[0047] In this embodiment, an image of a scene to be detected is acquired, and the image of the scene to be detected is input into a robot visual direction recognition model for robot navigation; the robot visual direction recognition model for robot navigation includes a mushroom body model and a continuous attractor network.

[0048] Step 102: Use the mushroom-shaped model to perform view direction analysis on the scene image to be detected, and generate multiple directional activity levels.

[0049] The mushroom body model includes multiple projection neurons, multiple Kenyon neurons, and multiple mushroom body output neurons.

[0050] Step 102 may include the following sub-steps S21-S23:

[0051] Step S21, using a plurality of projection neurons to perform image preprocessing on the scene image to be detected, and generating a plurality of normalized visual features;

[0052] Optionally, the process of performing image preprocessing on the scene image to be detected by the projection neuron includes the following sub-steps S1-S3:

[0053] Step S1, reducing the dimension of the scene image to be detected to generate a reduced-dimensional scene image;

[0054] Step S2, down-sampling the dimension-reduced scene image to generate a plurality of visual pixel points;

[0055] Step S3: normalize each visual pixel point to generate a normalized visual feature corresponding to each visual pixel point.

[0056] It should be noted that the scene image to be detected has two dimensions. The present invention first flattens the image into one dimension. The actual preprocessing of visual signals by insects may be complicated. According to the previous model, the architecture of the mushroom body model can be processed and learned on the image without complex feature extraction, only downsampling and normalization (Ardin et al., 2016; Dasgupta et al., 2017). Among them, each projection neuron performs image preprocessing operations on the scene image to be detected, so that each projection neuron outputs multiple normalized visual features.

[0057] Step S22, inputting the multiple normalized visual features into each Kenyon neuron for feature encoding, and generating multiple target Kenyon cell excitation levels;

[0058] Further, step S22 may include the following sub-steps S221-S223:

[0059] Step S221, sparsely encode each normalized visual feature through multiple Kenyon neurons to generate multiple visual codes;

[0060] Step S222: determining the initial Kenyon cell excitation level corresponding to each visual code based on each visual code;

[0061] Step S223, sorting the multiple initial Kenyon cell excitation levels in descending order, and selecting the initial Kenyon cell excitation level of the first preset number of digits as the corresponding target Kenyon cell excitation level.

[0062] It should be noted that Kenyon neurons compute a sparse representation of normalized visual features, which is used to model the connection from projection neurons (PNs) to Kenyon cells (KCs). The mapping from PNs to KCs can be viewed as a sparse, binary, randomly generated connection matrix W.

[0063] Specifically, each Kenyon neuron will sparsely encode multiple normalized input visual features, so that each Kenyon neuron outputs multiple visual codes, and based on each visual code, the initial Kenyon cell excitation level corresponding to each visual code is determined.

[0064] Furthermore, the multiple initial Kenyon cell excitation levels are sorted in descending order, and the initial Kenyon cell excitation levels of the first preset number are selected as the corresponding target Kenyon cell excitation levels; wherein the preset number can be set as needed, and the present invention is not limited thereto. For example, the preset number can be set to 20, then after the multiple initial Kenyon cell excitation levels are sorted in descending order, the first 20 initial Kenyon cell excitation levels are selected as the corresponding target Kenyon cell excitation levels.

[0065] Step S23: using the excitation levels of multiple target Kenyon cells as inputs to the output neurons of each mushroom body, and outputting multiple directional activity levels.

[0066] It should be noted that each mushroom body output neuron processes the excitation levels of multiple target Kenyon cells input, so that each mushroom body output neuron outputs multiple directional activity levels, where the output of the mushroom body model (directional activity level) is the likelihood of multiple categories of labels, which is input into the simplified central complex (CX) model simulated by the continuous attractor network; where the processing process of the directional activity level is specifically as follows:

[0067] y i =Wx i ;

[0068] Among them, y i is the directional activity level corresponding to the i-th visual encoding; x i is the i-th visual code, which is the visual code corresponding to the excitation level of the target Kenyon cell; W is the connection matrix, W∈R m×d , the matrix elements of W are binary elements, w ij ∈0,1, only η0d matrix elements take the value 1, η0 is the sampling rate, d is x i The dimension of ; m is the number of visual codes.

[0069] It is worth mentioning that, unlike typical multi-class classification datasets where there is no correlation between labels, the dataset in the present invention exhibits correlation between labels, allowing the output of "MBONs" to be interpolated to obtain finer directional resolution. In insects, multiple MBONs are connected to the fan-shaped body of the central complex (CX), that is, multiple MBONs are connected to a continuous attractor network. Since CANN (Continuous Attractor Network) has been shown to be a simplified model of CX, the present invention constructs a robot visual direction recognition model for robot navigation by modifying CANN to receive multiple outputs from "MBONs".

[0070] In this embodiment, a mushroom-shaped body model is used to perform view direction analysis on the scene image to be detected, and multiple directional activity levels are generated.

[0071] Step 103: Merge multiple directional activity levels through a continuous attractor network to generate a robot visual direction recognition result.

[0072] It should be noted that the continuous attractor network describes the ring attractor through multiple interconnected neurons. Each neuron is assigned a direction, stimulating distant neurons and inhibiting nearby neurons. Among them, the continuous attractor network is specifically:

[0073] ;

[0074] in, for The corresponding rate of change; is the excitation level of neurons at different positions at time t, After convergence, it is the robot visual direction recognition result; is the nerve density, , h is the number of direction discretization; For The excitatory connection from the neuron at position A to the neuron at position O; is the The firing rate of neurons at is the directional activity level, representing the input of the neuron at time t at o; is a constant used to control the magnitude of the cyclic interaction; is the half width of the excitatory connection range, with a typical value of 0.1; is the firing rate of the neuron at time t at o; is the degree of inhibition, with a typical value of 0.1; is the time constant of population dynamics, with a typical value of 1ms.

[0075] It is worth mentioning that the contribution of inhibitory connections is The output of the simplified mushroom body model is indirectly implemented by the division normalization in Feed to CANN (Continuous Attractor Network), where o corresponds to the label of “MBONs”, therefore, the present invention can add more neurons in CANN to interpolate to obtain finer resolution output.

[0076] Further, As the excitation level of different MBON neurons at time t, after passing through the continuous attractor network, the excitation level of the neurons corresponding to the angle of the robot is the highest, thereby determining the orientation of the robot.

[0077] Furthermore, based on the simplified mushroom-shaped body model for angle retrieval proposed in the present invention and a simplified central complex body model with a continuous attractor neural network (CANN), a robot visual direction recognition model for robot navigation is constructed. The former associates the view of an object with the direction angle. The latter combines multiple outputs of the former to provide detailed direction information (robot visual direction recognition results) for the robot's visual positioning. The test of the present invention on robot navigation proves the effectiveness of FlyOrien in data sets and practical applications, and shows that it is superior to traditional artificial neural networks in terms of efficiency.

[0078] Specifically, in order to simulate the behavior of animals finding directions, a two-part experiment was conducted on a quadruped robot for verification. In Experiment 1, the robot found a familiar object or landmark in the environment and made a 360° angle judgment around the landmark. In the second experiment, in an empty scene with no suitable objects or landmarks to circle around, the robot judged its own angle by rotating itself. A total of 360 images were collected for each sample, and the size of each image was 128x128 pixels. Compared with the test results of the redesigned dataset, the angle interval of the test set in this experiment changed from 10° to 1°, which is denser and more in line with the randomness of the angle and position of the robot in the real scene.

[0079] Conventional control methods rely on inertial odometers, the goal of this invention is to control the robot along a circular trajectory, facing the center of the circle. The typical radius of the circle is 0.5m, and the typical speed is 1-1.5deg / s. The onboard control system is remotely controlled by our server, communicating through the existing implemented Unitree protocol.

[0080] In the robot's visual positioning experiment of objects, the present invention uses robot motion control and image sampling algorithms to enable the robot to sample from different angles when rotating 360° around the object. The robot rotates around the object and takes a photo per degree with the head camera, taking a total of 360 photos. The binocular fisheye lens on the robot's head has a field of view of 180°. Through the robot's built-in official camera calibration algorithm, the corrected photos are transmitted in real time during the sampling process. The image size of the left and right eyes is 800×928. We will use the image from the left eye in subsequent experiments and compress it to 128×128. A set of sampled photos is selected every 10 degrees as the training data set, and the rest are used as the test set, with the nearest 10° as the label for the accuracy test. Cups, foam boxes, and plants were sampled and tested to verify the accuracy. The uncropped data set was also tested. According to the results, the accuracy of the first 5 angles of similarity predicted by the network exceeds 96%, and the accuracy of the first 2 angles exceeds 80%.

[0081] In the robot's visual localization experiment, the robot can only rotate in a circle by setting the speed. The robot was sampled in three different scenes: Laboratory 1, Laboratory 2, and Corridor. The sampling method was the same as the previous experiment. According to the results, the accuracy of the first 5 angles of similarity predicted by the network exceeded 96%, and the accuracy of the first 2 angles exceeded 80%. On the dense test set with an interval of 1°, the performance remained relatively stable.

[0082] For example, see Figure 2 ,When the scene image collected by the robot at 0° is first processed by the mushroom-shaped body model and the continuous attractor network, the robot visual direction recognition result output at this time indicates that the robot’s orientation at this time is 0°.

[0083] In this embodiment, multiple directional activity levels are merged through a continuous attractor network to generate a robot visual direction recognition result.

[0084] As a comparison of the technical effect, it can be combined with the existing technology for reference. In the natural environment, many clues can indicate the direction, such as the direction of the sun, the offset of the sunlight, the direction of the wind, and landmark buildings. Animals can use them for direction and navigation (Heinze, 2017). Most of these clues are perceived through vision. Even simple insects can use visual memory to remember the way home. After walking along the route once, it determines the route home by finding a familiar direction (Ardin et al., 2016). When memorizing directions, their lightweight neural circuits take less time than typical artificial neural networks (ANNs). Therefore, the present invention can achieve visual positioning of robots by studying the structure and learning rules of these circuits and retrieving the direction memory obtained by visual signals.

[0085] The intention of retrieving direction in the present invention is related to object pose estimation, but it is different in many aspects. The typical object pose estimation process uses a reference with planar features to build a planar mapping of the object, or uses a three-dimensional model for reference, such as a CAD model built for the object shape (Van de Ven et al., 2022). These references are accurate and have perfect memory. With these references, the pose of the object in the image can be estimated. These types of models have been widely used in fields such as face pose detection and industrial robot part picking (Fan et al., 2022). In the natural environment, objects with planar mappings are not common, and animals do not have predefined perfect memory for reference. For animals, in order to navigate on the ground, the relative direction of an object is more important than the pose of the object. Therefore, the robot visual direction recognition model for robot navigation proposed in the present invention focuses on relative direction rather than pose estimation.

[0086] Biological experiments and computational neuroscience models have shown that the MB of insects plays an important role in navigation tasks (Webb and Wystrach, 2016), such as visual homing. When desert ants leave their nests to forage, their pheromones evaporate at high temperatures. Therefore, they must rely on sky light and landmarks for navigation Ardin et al. (2016). Ardin et al. (2016) proposed a MB model in which ants can observe their surroundings and the MBON can indicate the unfamiliarity of the head direction based on visual input. According to the model, ants can learn the familiarity of the scene direction on the route out of the nest and then return home by searching for the most familiar head-on direction.

[0087] Another neural circuit that has been shown to be crucial in navigation tasks is the central complex (CX) (Honkane et al. 52, 2019). Neurons in the CX form a ring-shaped attractor that encodes information about head direction and homing direction in a group-coded manner. It can merge multiple directional cues to find a more accurate direction than a single cue could (Heinze, 2017). Neurons representing different directions integrate movement speed, and in this way, the CX can calculate and remember the direction and distance from the nest (Stone et al., 2017; Franconville et al., 2018). Computational neuroscience models predict that the neuronal activity matches the ring-shaped attractor, which is consistent with biological observations.

[0088] Based on the above foundation, the present invention combines a simplified mushroom-shaped body model for angle retrieval with a simplified central complex model with a continuous attractor neural network (CANN) (Wu et al., 2016) to obtain a robot visual direction recognition model for robot navigation. The former associates the view of an object with the direction angle. The latter merges multiple outputs of the former, thereby providing fine direction information for the robot's visual positioning. Among them, visual direction recognition helps navigation, especially in the absence of a reliable magnetic compass or GPS. Influenced by the brain's neural mechanism, the present invention combines a simplified mushroom-shaped body model for angle retrieval with a simplified central complex model with a continuous attractor neural network (CANN) (Wu et al., 2016) to obtain a robot visual direction recognition model for robot navigation, which provides multi-cue sensor integration, supports interpolation, and more refined direction representation through the continuous attractor neural network.

[0089] In an embodiment of the present invention, the present invention provides a robot visual direction recognition method for robot navigation. First, a scene image to be detected is obtained, and the scene image to be detected is input into a robot visual direction recognition model for robot navigation; the robot visual direction recognition model for robot navigation includes a mushroom model and a continuous attractor network; then, the mushroom model is used to perform view direction analysis on the scene image to be detected to generate multiple direction activity levels; finally, the multiple direction activity levels are merged through a continuous attractor network to generate a robot visual direction recognition result; based on the above scheme, the mushroom model and the continuous attractor network are used to process the acquired scene image to be detected, and the robot visual direction recognition result is output. The present invention can learn the familiarity of the scene direction through the mushroom model, and combine the continuous attractor network to merge multiple direction clues to form more comprehensive and accurate direction information. At the same time, the present invention can learn and infer the classification of its own direction based on the image information obtained by the robot's own camera in the robot body orientation recognition task, and output the orientation information through the integration of multiple classification labels, so as to obtain the robot body orientation angle without identifying feature points, thereby reducing the computational complexity.

[0090] In addition, most of the existing robot visual direction recognition methods are based on feature point recognition and comparison methods to infer the position of the camera and then calculate the direction of the robot. This method is highly dependent on local feature points and requires a large number of feature point comparisons. It has high computational complexity and low reliability when the texture is not rich. It also cannot directly use the global features of the image. The present invention does not require feature point comparison, the number of feature points does not affect the computational complexity, is not dependent on texture, and can directly use the global features of the image.

[0091] See also Figure 3 , Figure 3 A structural block diagram of a robot visual direction recognition system for robot navigation is provided in accordance with an embodiment of the present invention.

[0092] The present invention provides a robot visual direction recognition system for robot navigation, comprising:

[0093] An acquisition module 301 is used to acquire a scene image to be detected, and input the scene image to be detected into a robot visual direction recognition model for robot navigation; the robot visual direction recognition model for robot navigation includes a mushroom body model and a continuous attractor network;

[0094] Using module 302, for using the mushroom-shaped body model to perform view direction analysis on the image of the scene to be detected, and generating multiple directional activity levels;

[0095] The merging module 303 is used to merge multiple directional activity levels through a continuous attractor network to generate a robot visual direction recognition result.

[0096] Further, the mushroom body model includes a plurality of projection neurons, a plurality of Kenyon neurons, and a plurality of mushroom body output neurons; module 302 is adopted, including:

[0097] The first submodule is used to use a plurality of projection neurons to perform image preprocessing on the scene image to be detected, and generate a plurality of normalized visual features;

[0098] The second submodule is used to input multiple normalized visual features into each Kenyon neuron for feature encoding, and generate multiple target Kenyon cell excitation levels;

[0099] The third submodule is used to use the excitation levels of multiple target Kenyon cells as inputs of each mushroom body output neuron, and output multiple directional activity levels.

[0100] Furthermore, the first submodule is specifically used for:

[0101] Perform dimensionality reduction on the scene image to be detected to generate a dimensionality-reduced scene image;

[0102] Downsample the reduced-dimensional scene image to generate multiple visual pixels;

[0103] Each visual pixel is normalized respectively to generate a normalized visual feature corresponding to each visual pixel.

[0104] Furthermore, the second submodule is specifically used for:

[0105] Each normalized visual feature is sparsely encoded through multiple Kenyon neurons to generate multiple visual codes;

[0106] Based on each visual code, determine the initial Kenyon cell excitation level corresponding to each visual code;

[0107] The multiple initial Kenyon cell excitation levels are sorted in descending order, and the initial Kenyon cell excitation level of the first preset number of digits is selected as the corresponding target Kenyon cell excitation level.

[0108] Furthermore, the processing process of the directional activity level is as follows:

[0109] y i =Wx i ;

[0110] Among them, y i is the directional activity level corresponding to the i-th visual encoding; x i is the i-th visual encoding; W is the connection matrix, W∈Rm×d , the matrix elements of W are binary elements, w ij ∈0,1, only η0d matrix elements take the value 1, η0 is the sampling rate, d is x i The dimension of ; m is the number of visual codes.

[0111] Furthermore, the continuous attractor network is specifically:

[0112] ;

[0113] in, for The corresponding rate of change; is the excitation level of neurons at different positions at time t, After convergence, it is the robot visual direction recognition result; is the nerve density, , h is the number of direction discretization; For The excitatory connection from the neuron at position A to the neuron at position O; is the The firing rate of neurons at is the directional activity level, representing the input of the neuron at time t at o; is a constant used to control the magnitude of the cyclic interaction; is the half width of the excitatory connection range, with a typical value of 0.1; is the firing rate of the neuron at time t at o; is the degree of inhibition, with a typical value of 0.1; is the time constant of population dynamics, with a typical value of 1ms.

[0114] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described systems, modules and sub-modules can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0115] An embodiment of the present invention further provides a computer device, including a memory and a processor, wherein a computer program is stored in the memory; when the computer program is executed by the processor, the processor executes the steps of the robot visual direction recognition method for robot navigation as described in the first embodiment above.

[0116] An embodiment of the present invention further provides a computer-readable storage medium having a computer program / instruction stored thereon. When the computer program / instruction is executed by a processor, the steps of the robot visual direction recognition method for robot navigation as described in the first embodiment above are implemented.

[0117] An embodiment of the present invention further provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the steps of the robot visual direction recognition method for robot navigation as described in the first embodiment above.

[0118] In the several embodiments provided in the present application, it should be understood that the disclosed systems and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0119] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0120] As described above, the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features thereof may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A robot visual direction recognition method for robot navigation, characterized in that: include: Acquire a scene image to be detected, and input the scene image to be detected into a robot visual direction recognition model for robot navigation; the robot visual direction recognition model for robot navigation includes a mushroom body model and a continuous attractor network; Using the mushroom-shaped body model to perform view direction analysis on the scene image to be detected, and generating multiple directional activity levels; The plurality of directional activity levels are merged through the continuous attractor network to generate a robot visual direction recognition result.

2. The robot visual direction recognition method for robot navigation according to claim 1, characterized in that: The mushroom body model includes a plurality of projection neurons, a plurality of Kenyon neurons, and a plurality of mushroom body output neurons; the mushroom body model is used to perform view direction analysis on the scene image to be detected to generate a plurality of directional activity levels, including: Using a plurality of projection neurons to perform image preprocessing on the scene image to be detected, respectively, to generate a plurality of normalized visual features; Inputting the plurality of normalized visual features into respective Kenyon neurons for feature encoding to generate a plurality of target Kenyon cell excitation levels; The excitation levels of the target Kenyon cells are respectively used as inputs of the mushroom body output neurons to output multiple directional activity levels.

3. The robot visual direction recognition method for robot navigation according to claim 2, characterized in that: The process of the projection neuron performing image preprocessing on the scene image to be detected includes: Performing dimensionality reduction on the scene image to be detected to generate a dimensionality-reduced scene image; Down-sampling the dimension-reduced scene image to generate a plurality of visual pixel points; Each of the visual pixel points is normalized to generate a normalized visual feature corresponding to each of the visual pixel points.

4. The robot visual direction recognition method for robot navigation according to claim 2, characterized in that: The plurality of normalized visual features are respectively inputted into each Kenyon neuron for feature encoding to generate a plurality of target Kenyon cell excitation levels, including: Sparsely encoding each of the normalized visual features through a plurality of Kenyon neurons to generate a plurality of visual codes; Based on each of the visual codes, determining the initial Kenyon cell excitation level corresponding to each of the visual codes; The multiple initial Kenyon cell excitation levels are sorted in descending order, and the initial Kenyon cell excitation levels of the first preset number of digits are selected as the corresponding target Kenyon cell excitation levels.

5. The robot visual direction recognition method for robot navigation according to claim 2, characterized in that: The processing process of the directional activity level is specifically as follows: y i =Wx i ; Among them, y i is the directional activity level corresponding to the i-th visual encoding; x i is the i-th visual encoding; W is the connection matrix, W∈R m×d , the matrix elements of W are binary elements, w ij ∈0,1, only η0d matrix elements take the value 1, η0 is the sampling rate, d is x i The dimension of ; m is the number of visual codes.

6. The robot visual direction recognition method for robot navigation according to claim 1, characterized in that: The continuous attractor network is specifically: ； in, for The corresponding rate of change; is the excitation level of neurons at different positions at time t, After convergence, it is the robot visual direction recognition result; is the nerve density, , h is the number of direction discretization; For The excitatory connection from the neuron at position A to the neuron at position O; is the The firing rate of neurons at is the directional activity level, representing the input of the neuron at time t at o; is a constant used to control the magnitude of the cyclic interaction; is the half width of the excitatory connection range, with a typical value of 0.1; is the firing rate of the neuron at time t at o; is the degree of inhibition, with a typical value of 0.1; is the time constant of population dynamics, with a typical value of 1ms.

7. A robot visual direction recognition system for robot navigation, characterized in that: include: An acquisition module, used for acquiring a scene image to be detected, and inputting the scene image to be detected into a robot visual direction recognition model for robot navigation; the robot visual direction recognition model for robot navigation includes a mushroom body model and a continuous attractor network; An adopting module, configured to use the mushroom-shaped body model to perform view direction analysis on the scene image to be detected, and generate multiple directional activity levels; A merging module is used to merge the multiple directional activity levels through the continuous attractor network to generate a robot visual direction recognition result.

8. A computer device, characterized in that: It comprises a memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the processor executes the steps of the robot visual direction recognition method for robot navigation as described in any one of claims 1-6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed, the robot visual direction recognition method for robot navigation as described in any one of claims 1 to 6 is implemented.

10. A computer program product, characterized in that The computer program product includes a computer program stored on a non-transitory computer-readable storage medium, and the computer program includes program instructions, wherein when the program instructions are executed by a computer, the computer executes the robot visual direction recognition method for robot navigation as described in any one of claims 1-6.

Images