Target-driven visual semantic navigation methods, storage media, and devices for indoor scenarios

Abstract

This invention provides a target-driven visual semantic navigation method, storage medium, and device for indoor scenes. The method includes: analyzing the size distribution of target objects in indoor scene data; performing sample augmentation on images of objects with tail-sized dimensions; training a target detector based on the augmented images of objects with tail-sized dimensions; constructing a target-driven visual semantic navigation network model based on the target detector and a neural network; training the visual semantic navigation network model; determining the target-driven visual semantic navigation steps; and completing the visual semantic navigation task based on the visual semantic navigation steps and the trained visual semantic navigation network model. This invention can assist robots in navigating indoor scenes, improve their scene perception capabilities, and help indoor mobile robots perform various tasks.

Description

Technical Field

[0001] This invention relates to the field of scene perception and navigation technology, and in particular to the field of indoor scene perception and navigation technology based on active vision. Background Technology

[0002] With the aging population and the optimization and upgrading of the labor force structure, labor costs are increasing year by year, and more and more industries are turning their attention to robot-led solutions. Replacing humans with robots for simple, repetitive, and low-skilled tasks is an effective method and a future trend for liberating labor. Robots in indoor environments require good target recognition and navigation capabilities, demanding a high degree of autonomy. To enable them to handle more complex tasks and provide more human-centered services, their ability to perceive their surroundings needs to be improved. Therefore, researching robot navigation methods in various scenarios has practical significance and application value.

[0003] To achieve visual navigation of robots in indoor scenes, current methods mainly include 3D scene measurement, semantic mapping, and path planning. These methods typically require the pre-establishment of a 3D scene map and provide reliable localization for map and path tracking. This allows them to perform well in known scenes, but their navigation effectiveness declines when the scene changes or in similar unknown environments. To overcome these limitations, data-driven machine learning-based navigation is emerging. This approach directly learns the logical mapping between raw visual perception information and robot actions in an end-to-end manner, forming a new paradigm of semantic visual navigation. In this new paradigm, navigation is map-independent and can leverage prior navigation experience even in new, similar environments.

[0004] A review of publicly available literature on existing technologies revealed that current visual-semantic navigation methods for target-driven environments in indoor settings only use current-moment data for action decisions, neglecting the influence of historical scene information and corresponding action information on the current decision. Furthermore, during the training process of the navigation model, the model output contains numerous illegal actions, rendering training examples invalid and resulting in low data utilization. Moreover, the imaging quality of the target object within the observer's field of view is not considered during navigation, causing the detector to become insensitive to the target object after the robot approaches it. Summary of the Invention

[0005] In view of the shortcomings of the prior art described above, the purpose of this invention is to provide a target-driven visual semantic navigation method, storage medium and device for indoor scenes, so as to solve the technical problem of poor visual semantic navigation effect under target-driven conditions in indoor environments in the prior art.

[0006] To achieve the above and other related objectives, this invention provides a target-driven visual semantic navigation method for indoor scenes. The method includes: analyzing the size distribution of target objects in indoor scene data; performing sample augmentation on images of objects with tail sizes; training a target detector based on the augmented images of objects with tail sizes; constructing a target-driven visual semantic navigation network model based on the target detector and a neural network; training the visual semantic navigation network model; determining the target-driven visual semantic navigation steps; and completing the visual semantic navigation task based on the visual semantic navigation steps and the trained visual semantic navigation network model.

[0007] In one embodiment of the present invention, the analysis of the size distribution of target objects in indoor scene data includes: counting the number of detection boxes for large, medium and small objects in the scene data, as well as the number of images containing large, medium and small objects, and analyzing their respective proportions to obtain the size distribution of the dataset, and determining the head-sized object images with the largest proportion and the tail-sized object images with the smallest proportion.

[0008] In one embodiment of the present invention, the sample enhancement of the tail-sized object image includes: generating a tail-sized object image using a head-sized object image; generating a smaller tail-sized object image from a larger head-sized object image using an image reduction and stitching method; and generating a larger tail-sized object image from a smaller head-sized object image using an adaptive local cropping and magnification method.

[0009] In one embodiment of the present invention, training the target detector based on the enhanced tail-size object image includes: constructing a dataset final state based on the enhanced tail-size object image; constructing a series of dataset transition states between the dataset final state and the dataset initial state; and selecting different dataset transition states to train the target detector in each training iteration.

[0010] In one embodiment of the present invention, in the target-driven visual semantic navigation network model constructed based on the target detector and the neural network: the input is: the scene information and corresponding action information of the three previous moments based on the current moment, which are used as historical moment information for input; the output is: the action with the highest probability score in the six-dimensional action space of forward, backward, left, right, clockwise rotation, and counterclockwise rotation.

[0011] In one embodiment of the present invention, in the construction of the target-driven visual semantic navigation network model based on the target detector and the neural network: the current scene information, target rectangle bounding box, and action space information feature quantities are extracted using a long short-term memory neural network and a convolutional neural network, the mapping relationship between the above feature quantities and the output is constructed, and a corresponding visual semantic navigation model is established.

[0012] In one embodiment of the present invention, training the visual semantic navigation network model includes: using a preset step size as a training batch; if, within a training batch, the distance information between the robot and the target object is less than a distance metric threshold and the detection score of the object within the field of view is higher than a score threshold, it indicates that the robot has completed the navigation task and begins training for the next batch; otherwise, the robot repeats the above decision-making and movement process according to the navigation strategy until the robot meets the above task termination condition or the decision step size reaches the preset step size; after a series of action attempts in each batch are completed, the corresponding reward factor is evaluated, and the weights of the visual semantic navigation network model are updated based on the reward factor, so that the visual semantic navigation network model can learn the corresponding navigation strategy.

[0013] In one embodiment of the present invention, the target-driven visual semantic navigation step includes: the visual semantic navigation network model receiving an RGB-D image of the scene, evaluating the observation angle and distance of the target, generating an action decision, adjusting the robot's position, and repeating the above decision process at the new position until the robot reaches the vicinity of the target or the number of decision attempts reaches the upper limit.

[0014] To achieve the above and other related objectives, the present invention also provides a storage medium storing program instructions, wherein when the program instructions are executed, they implement the steps of the target-driven visual semantic navigation method in an indoor scene as described above.

[0015] To achieve the above and other related objectives, the present invention also provides an electronic device, including a memory for storing a computer program and a processor for running the computer program to implement the steps of the target-driven visual semantic navigation method in an indoor scene as described above.

[0016] To achieve the above and other related objectives, the present invention also provides a mobile robot that uses the electronic equipment described above.

[0017] As described above, the target-driven visual semantic navigation method, storage medium, and device for indoor scenes of the present invention have the following beneficial effects:

[0018] This invention analyzes the relationship between the imbalance in object size distribution in scene data and the differences in the model's detection capabilities for objects of different sizes. It designs a multi-size target adaptive image enhancement method to enhance images of objects of varying sizes. Based on this method, a dynamic balancing method for object size distribution in the dataset is designed and implemented. This method dynamically adjusts the distribution of objects of different sizes within the dataset during training to enhance the model's supervision of tail-sized objects. While maintaining the model's detection performance for head-sized objects, it improves its detection capability for tail-sized objects. Simultaneously, a visual semantic navigation network model is constructed based on Conv-LSTM and CNN networks. An optimized LSTM initialization method and an action space-optimized model training method are established, enabling the model to directly generate navigation strategies based on scene semantic context information, independent of scene map modeling. Furthermore, navigation strategies built in known scenes can be extended to similar unknown scenes. This invention can assist robots in navigating indoor scenes, improving their scene perception capabilities and providing assistance for indoor mobile robots to perform various tasks. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 The flowchart shown is a target-driven visual semantic navigation method for indoor scenes according to the present invention.

[0021] Figure 2 The diagram illustrates the multi-size image adaptive local cropping method of this invention.

[0022] Figure 3 This is shown as the object size distribution dynamic balance training method of the present invention.

[0023] Figures 4(a) and 4(b) show the changes in scene data during the dynamic balance training process of the present invention.

[0024] Figure 5 This is shown as the Visual Semantic Navigation Model (VSNN) constructed in this invention.

[0025] Figures 6(a) and 6(b) show the model training and verification processes of the present invention.

[0026] Figure 7 The diagram shown is a schematic block diagram of an electronic device according to an embodiment of this application.

[0027] Component designation explanation

[0028] 101 Electronic Devices

[0029] 1001 processor

[0030] 1002 Memory

[0031] S100~S600 Steps Detailed Implementation

[0032] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that, unless otherwise specified, the following embodiments and features described therein can be combined with each other.

[0033] The purpose of this embodiment is to provide a target-driven visual semantic navigation method, storage medium, and device for indoor scenes, to solve the technical problem of poor performance of target-driven visual semantic navigation in indoor environments in existing technologies. The target-driven visual semantic navigation method of this embodiment can solve navigation problems in complex indoor scenes. The method ensures efficient model training and adjusts the observation perspective while navigating, maintaining a good observation angle while approaching the target object. Furthermore, the method does not rely on the construction of a scene map and can perform well even in similar but unknown environments.

[0034] This embodiment provides a target-driven visual semantic navigation method for indoor scenes, including: a multi-size object image enhancement method; a dynamic balancing training method for object size distribution in the dataset; a target-driven visual semantic navigation model built based on Long Short-Term Memory (LSTM) networks and Convolutional Neural Networks (CNNs); an LSTM initialization method based on reverse reasoning and a training method based on action space optimization; and the general steps of target-driven visual semantic navigation. This embodiment can assist robots in navigating indoor scenes, improve their scene perception capabilities, and help indoor mobile robots perform various tasks.

[0035] The following will describe in detail the principles and implementation methods of the target-driven visual semantic navigation method, storage medium, and device for indoor scenes of the present invention, so that those skilled in the art can understand the target-driven visual semantic navigation method, storage medium, and device for indoor scenes of the present invention without creative effort.

[0036] Example 1

[0037] like Figure 1 As shown, the target-driven visual semantic navigation method for indoor scenes in this embodiment includes:

[0038] Step S100: Analyze the size distribution of target objects in the indoor scene data;

[0039] Step S200: Perform sample augmentation on the image of the object with tail size;

[0040] Step S300: Train the target detector based on the enhanced tail-size object image;

[0041] Step S400: Construct a target-driven visual semantic navigation network model based on the target detector and the neural network;

[0042] Step S500: Train the visual semantic navigation network model;

[0043] Step S600: Determine the target-driven visual semantic navigation steps, and complete the visual semantic navigation task based on the visual semantic navigation steps and the trained visual semantic navigation network model.

[0044] The following combination Figures 2 to 6(b) The steps S100 to S600 of the target-driven visual semantic navigation method in the indoor scene of this embodiment will be described in detail.

[0045] Step S100: Analyze the size distribution of target objects in the indoor scene data.

[0046] In this embodiment, the analysis of the size distribution of target objects in indoor scene data includes: counting the number of detection boxes for large, medium and small objects in the scene data, as well as the number of images containing large, medium and small objects, and analyzing their respective proportions to obtain the size distribution of the dataset, and determining the head-sized object images with the largest proportion and the tail-sized object images with the smallest proportion.

[0047] Step S200: Perform sample enhancement on the image of the object with tail size.

[0048] In this embodiment, the sample enhancement of the tail-sized object image includes: generating a tail-sized object image using a head-sized object image; generating a smaller tail-sized object image from a larger head-sized object image using image downsizing and stitching; and generating a larger tail-sized object image from a smaller head-sized object image using an adaptive local cropping and magnification method.

[0049] Specifically, in this embodiment, when converting a larger image to a smaller image, the four large images are reduced to one-quarter of the original image using neighborhood interpolation and then stitched together. At the same time, the targets in each image are re-labeled using rectangular bounding boxes. When converting a smaller image to a larger image, a multi-size target adaptive local cropping method is used. That is, the image is divided into sub-regions, the intersection of the target with each segmentation line is examined, and the target is assigned to different regions according to different intersection conditions. Finally, the sub-image containing the target is enlarged to the original image size using cubic spline interpolation to complete the enhancement process.

[0050] Step S300: Train the target detector based on the enhanced tail-size object image.

[0051] In this embodiment, training the target detector based on the enhanced tail-size object image includes: constructing the final state of the dataset based on the enhanced tail-size object image; constructing a series of dataset transition states between the final state and the initial state of the dataset; and selecting different dataset transition states to train the target detector in each training iteration.

[0052] In this embodiment, the final state of the scene dataset is first determined. Using the aforementioned tail-size object image sample enhancement method, a series of transition states are constructed between the initial and final states of the dataset. In each iteration of training, different dataset transition states are used to train the target detector.

[0053] Step S400: Construct a target-driven visual semantic navigation network model based on the target detector and the neural network.

[0054] In this embodiment, the visual semantic navigation network model driven by the target detector and the neural network is constructed as follows: the input is the scene information and corresponding action information of the three previous moments based on the current moment, which are used as historical moment information; the output is the action with the highest probability score in the six-dimensional action space of forward, backward, left, right, clockwise rotation, and counterclockwise rotation.

[0055] In this embodiment, the visual semantic navigation network model driven by the target detector and the neural network is constructed by: using the feature quantities of the current scene information, target rectangle bounding box, and action space information extracted by the long short-term memory neural network and the convolutional neural network, constructing the mapping relationship between the above feature quantities and the output, and establishing the corresponding visual semantic navigation model.

[0056] In this embodiment, the model output is the action with the highest probability score in the six-dimensional action space of forward, backward, left, right, clockwise rotation, and counterclockwise rotation. In the design of the model input, the historical correlation of action decision is considered. Scene information and corresponding action information from the three previous historical moments are introduced. Conv-LSTM is used to extract relevant features, and CNN is used to extract the scene information, target rectangle bounding box, and action space features at the current moment. The mapping relationship between the above features and the output is constructed to establish the corresponding visual semantic navigation model.

[0057] Step S500: Train the visual semantic navigation network model.

[0058] In this embodiment, training the visual semantic navigation network model includes: using a preset step size (e.g., preset step size T=5) as a training batch; if, within a training batch, the distance information between the robot and the target object is less than a distance metric threshold (e.g., 700) and the detection score of the object within the field of view is higher than a score threshold, it indicates that the robot has completed the navigation task and begins training for the next batch; otherwise, the robot repeats the above decision-making and movement process according to the navigation strategy until the robot meets the above task termination condition or the decision step size reaches the preset step size; after the series of action attempts in each batch are completed, the corresponding reward factor is evaluated, and the weights of the visual semantic navigation network model are updated based on the reward factor, so that the visual semantic navigation network model can learn the corresponding navigation strategy.

[0059] Specifically, in this embodiment, the training of the visual semantic navigation model includes the following steps:

[0060] 1) In the initial stage of the task, there will be missing historical information, requiring the initialization of the LSTM layer. A reverse reasoning approach is used to infer possible historical scene information from the current scene information, filling in the missing historical information and completing the LSTM initialization. Since navigation tasks often involve approaching the target from a distance, the inference here prioritizes backward actions. If backward action is deemed illegal, other legal actions are selected from the action space.

[0061] 2) To reduce the generation of illegal actions by the network, which leads to excessive ineffective training, reduced data utilization, and low model efficiency, the model needs to evaluate actions after making action decisions. When an illegal action is detected, it is replaced by another legal action in the action space. The forward action is given priority. If the forward action is illegal, another legal action is selected from the action space.

[0062] 3) To enable the model to learn the required strategies, reinforcement learning is used to train the model. The reward factor is constructed using the target detector's score for the target and the target's distance from the observer as evaluation metrics. The model is given the current input to make action decisions, obtains the input for the next step, and is evaluated using the aforementioned evaluation metrics. A training batch consists of 5 steps. If the robot reaches a position within 5 steps, and the target object detection score and distance metric at that position meet the threshold requirements, the robot has successfully completed the navigation task. At this point, the reward factor is calculated to provide positive reinforcement to the model, and the next batch of training begins. Otherwise, the robot continues the evaluation, decision-making, and movement process until the task is completed or the decision step size reaches a predetermined upper limit. The reward factor is then calculated to update the model.

[0063] Step S600: Determine the target-driven visual semantic navigation steps, and complete the visual semantic navigation task based on the visual semantic navigation steps and the trained visual semantic navigation network model.

[0064] In this embodiment, the target-driven visual semantic navigation step includes: the visual semantic navigation network model receives an RGB-D image of the scene, evaluates the observation angle and distance of the target, generates an action decision, adjusts the robot's position, and repeats the above decision-making process at the new position until the robot reaches the vicinity of the target or the number of decision-making attempts reaches the upper limit.

[0065] The general steps of target-driven visual semantic navigation are as follows: acquire an RGB-D image of the current scene, and evaluate the viewing angle and distance of the target. If the task termination condition is met, it means that the robot has moved to the vicinity of the target object. Otherwise, the model generates action decisions, allowing the robot to move in the scene to approach the object.

[0066] The following example illustrates the target-driven visual semantic navigation method in an indoor scene according to this embodiment.

[0067] 1. First, analyze the distribution of objects of different sizes in the scene data to determine the head and tail sizes. Taking the Active Vision Dataset as an example, its size distribution is shown in the initial distribution in Table 1. It can be confirmed that the head size is a medium-sized object and the tail size is a large-sized object. Therefore, medium-sized objects are used to enhance the images of large and small objects.

[0068] Table 1: Distribution of Object Sizes in Scene Data

[0069]

[0070] 2. When performing image enhancement, to generate a small-sized object image from a medium-sized object image, image stitching is used. This involves reducing four medium-sized object images to one-quarter of their original size and then stitching them together into a single image. When generating a large-sized object image from a medium-sized object image, an adaptive local cropping method is used, such as... Figure 2 As shown, the original image is divided into 5 regions. The intersections of the target and the dividing lines are examined. Targets that intersect with dividing lines l1 and l2 are assigned to region S1, targets that intersect with dividing lines l3 and l4 are assigned to region S5, and targets that intersect with dividing lines l1, l2, l3, and l4 are assigned to region S5. Each region is scaled proportionally to ensure the region fully includes the target. Simultaneously, sub-regions containing the target are enlarged to the original image size, completing the enhancement process.

[0071] 3. Using Faster-RCNN as the backbone network, it is trained using a dynamic equilibrium training method for object size distribution. The corresponding process is as follows: Figure 3 As shown in Table 1, the data size distribution after processing is enhanced, and the changes in object size distribution during training are shown in Figures 4(a) and 4(b). To evaluate the performance of the method, Table 2 provides a comparison of different models, showing that the method proposed in this invention improves the model's detection capability.

[0072] Table 2 Model Performance Comparison

[0073]

[0074] 4. Construct a visual semantic navigation model, such as Figure 5 As shown. Specifically, it includes: the model output is a certain action in a six-dimensional action space, such as forward, backward, left, right, clockwise rotation, and counterclockwise rotation; in the model input design, the scene information and corresponding action information of the three previous time moments are taken to form historical time information, which is extracted using Conv-LSTM, and the current time moment information is formed by taking the current scene RGB-D image information, target rectangle bounding box, and action space information, which is then extracted using CNN.

[0075] 5. The training process of the visual semantic navigation model is shown in Figure 6(a). The model acquires the corresponding RGB-D image of each scene, moves according to the action decisions generated by the navigation strategy, and evaluates the observation difficulty and distance of the target. During training, a training batch is defined as a step size T = 5. If, within a training batch, the distance between the robot and the target object is less than the distance metric threshold of 700 and the object's detection score within the field of view is higher than 0.5, then the robot has completed the navigation task and can directly begin training for the next batch. Otherwise, the robot repeats the above decision-making and movement process according to the navigation strategy until the robot meets the above task termination condition or the decision step size reaches T. After each batch of action attempts, the corresponding reward factors are evaluated according to equations (1-1) and (1-2), and the model weights are updated to enable the robot to learn the corresponding navigation strategy. In this embodiment, β1 = 0.3, β2 = 0.7, and α = 2500. During training, an LSTM initialization method based on reverse reasoning is needed to fill in the missing historical scene information, and the effectiveness of the action decisions generated by the model is supervised. An optimization method based on action space is used for training.

[0076] The navigation task was tested, and the process is shown in Figure 6(b). An RGB-D image of the scene was acquired at the current moment and evaluated. If the distance metric between the robot and the target was less than 700, it meant the robot had moved to the vicinity of the target. Otherwise, the model generated an action decision, instructing the robot to move within the scene and approach the target object. The actual effects of different navigation decisions in the training and testing scenarios are shown in Tables 3 and 4.

[0077] Table 3 Comparison of Navigation Decision Performance (Training Set)

[0078] Strategy Subset 1 Subset 2 Subset 3 average Random 0.242 0.248 0.239 0.243 Forward 0.175 0.194 0.177 0.182 PG 0.420 0.367 0.385 0.391 AODN 0.532 0.577 0.537 0.549 VSNN 0.685 0.637 0.550 0.636

[0079] Table 4 Comparison of Navigation Decision Performance (Test Set)

[0080]

[0081] As can be seen from the above, this embodiment fully considers the extremely unbalanced distribution of object sizes in the scene data. Through a dynamic size balancing training method, the detector can better detect objects with large tail sizes. At the same time, a visual semantic navigation network is constructed, which can directly build navigation strategies based on the semantic context information of the scene, without relying on the construction of a scene map, and can also perform well in unknown similar scenes.

[0082] Example 2

[0083] like Figure 7As shown, this embodiment provides an electronic device 101, which includes a processor 1001 and a memory 1002. The memory 1002 is used to store computer programs. The processor 1001 is used to execute the computer programs stored in the memory 1002, so that the electronic device 101 performs the steps of the target-driven visual semantic navigation method in an indoor scene as described in Embodiment 1. Since the specific implementation process of the steps of the target-driven visual semantic navigation method in an indoor scene has been described in detail in Embodiment 1, it will not be repeated here.

[0084] Processor 1001 is a Central Processing Unit (CPU). Memory 1002 is connected to processor 1001 via a system bus and communicates with it. Memory 1002 stores computer programs, and processor 1001 runs the computer programs to execute the target-driven visual semantic navigation method for indoor scenes. Memory 1002 may include random access memory (RAM) and may also include non-volatile memory, such as at least one disk storage device.

[0085] Furthermore, this embodiment also provides a computer-readable storage medium storing a computer program thereon. When executed by the processor 1001, the computer program implements the steps in the target-driven visual semantic navigation method for indoor scenes described in Embodiment 1. Embodiment 1 has already provided a detailed description of the target-driven visual semantic navigation method for indoor scenes, and will not be repeated here.

[0086] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented using computer program-related hardware. The aforementioned computer program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0087] In summary, this invention analyzes the relationship between the imbalance in object size distribution in scene data and the differences in the model's detection capabilities for objects of different sizes. It designs a multi-size target adaptive image enhancement method to enhance images of objects of different sizes. Based on this method, a dynamic balancing method for object size distribution in the dataset is designed and implemented. This method dynamically adjusts the distribution of objects of different sizes within the dataset during training to enhance the model's supervision of tail-sized objects. While maintaining the model's detection performance for head-sized objects, it improves its detection capability for tail-sized objects. Simultaneously, a visual semantic navigation network model is constructed based on Conv-LSTM and CNN networks. An optimized LSTM initialization method and an action space-optimized model training method are established, enabling the model to directly generate navigation strategies based on scene semantic context information, independent of scene map modeling. Furthermore, navigation strategies built in known scenes can be extended to similar unknown scenes. This invention can assist robots in navigating indoor scenes, improving their scene perception capabilities and providing assistance for indoor mobile robots to perform various tasks. Therefore, this invention effectively overcomes the various shortcomings of existing technologies and has high industrial application value.

[0088] The above embodiments are merely illustrative of the principles and effects of the present invention and are not intended to limit the invention. Any person skilled in the art can modify or alter the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or alterations made by those skilled in the art without departing from the spirit and technical concept disclosed in the present invention should still be covered by the claims of the present invention.

Claims

1. A target-driven visual semantic navigation method in an indoor scene, characterized in that: The method comprises: Analyzing the size distribution of the target object in the indoor scene data, including: counting the number of detection boxes of large, medium and small size objects in the scene data, and the number of images containing large, medium and small size objects, and analyzing their respective proportions to obtain the size distribution of the data set, and determine the head size object image with the highest proportion and the tail size object image with the lowest proportion; Sample enhancement is performed on the tail size object image, including: generating a tail size object image using a head size object image: for a larger size head size object image, a smaller size tail size object image is generated using image reduction splicing; for a smaller size head size object image, a larger size tail size object image is generated using adaptive local interception magnification; Training a target detector based on the enhanced tail size object image; Building a visual semantic navigation network model under target driving based on the target detector and neural network; Training the visual semantic navigation network model; Determine the visual semantic navigation steps under the target driving, and complete the visual semantic navigation task based on the visual semantic navigation steps and the trained visual semantic navigation network model. 2.The target-driven visual semantic navigation method in an indoor scene according to claim 1, characterized in that: The training of the target detector based on the enhanced tail size object image comprises: Building a data set terminal state based on the enhanced tail size object image; A series of data set transition states are constructed between the data set terminal state and the data set initial state, and in each training iteration, different data set transition states are selected to train the target detector. 3.The target-driven visual semantic navigation method in an indoor scene according to claim 1, characterized in that: In the construction of the visual semantic navigation network model under the target driving based on the target detector and neural network: The input is: the scene information and the corresponding action information of the previous three time points based on the current time as the reference, which is input as historical time information; The output is: the action with the maximum probability score in the six-dimensional action space of forward, backward, left, right, clockwise rotation and counterclockwise rotation.

4. The target-driven visual semantic navigation method in indoor scenes according to claim 3, characterized in that: In the construction of the visual semantic navigation network model under the target driving based on the target detector and neural network: The features of the current time scene information, target rectangular bounding box and action space information are extracted using long short-term memory neural network and convolutional neural network, the mapping relationship between the above features and the output is constructed, and the corresponding visual semantic navigation model is established.

5. The method of claim 1, wherein the target-driven visual semantic navigation in indoor scenes is characterized by: The training of the visual semantic navigation network model comprises: The preset step length is one training batch; if the distance information between the robot and the target object is less than the distance metric threshold and the detection score of the object in the field of view is higher than the score threshold within one training batch, it indicates that the robot has completed the navigation task, and the next batch of training is started; otherwise, the robot repeats the decision-making and movement process according to the navigation strategy until the robot meets the above task end condition or the decision-making step length reaches the preset step length; after a series of action attempts in each batch end, the corresponding reward factor is evaluated, and the weights of the visual semantic navigation network model are updated based on the reward factor, so that the visual semantic navigation network model can learn the corresponding navigation strategy.

6. The target-driven visual semantic navigation method in indoor scenes according to claim 1, characterized in that: The visual semantic navigation step driven by the target includes: The visual semantic navigation network model receives the RGB-D image of the scene, evaluates the observation angle and distance of the target, makes an action decision, adjusts the position of the robot, and repeats the decision-making process in the new position until the robot reaches the vicinity of the target or the number of decision-making reaches the upper limit.

7. A storage medium storing program instructions, wherein, The program instructions, when executed, implement the steps of the target-driven visual semantic navigation method in an indoor scene as claimed in any one of claims 1 to 6.

8. An electronic device, comprising: The program instructions, when executed, implement the steps of the target-driven visual semantic navigation method in an indoor scene as claimed in any one of claims 1 to 6. The program instructions, when executed, implement the steps of the target-driven visual semantic navigation method in an indoor scene as claimed in any one of claims 1 to 6.

Images