Position prediction model training method and device and computer equipment

Abstract

The invention relates to a position prediction model training method and device, computer equipment, a computer readable storage medium and a computer program product. The method comprises the following steps: acquiring a target pixel position of a target reference object in a reference image and a real physical position determined by a physical reference object in a physical space based on the position of a robot; inputting the target pixel position into a to-be-trained position prediction model for processing to obtain a predicted physical position corresponding to the physical reference object; on the basis of the difference between the predicted physical position and the real physical position, updating model parameters of the position prediction model to obtain an updated position prediction model; and re-inputting the target pixel position into the updated position prediction model for processing until a trained position prediction model is obtained. By adopting the method, the distance between the robot and the physical object can be accurately and efficiently measured.

Description

Technical Field

[0001] The present application relates to the field of robot vision technology, and in particular to a training method, apparatus, computer equipment, computer-readable storage medium, and computer program product for a position prediction model. Background Art

[0002] In the field of robot vision technology, in application scenarios based on monocular vision ranging, the distance of the object in front is determined by establishing the relationship between the two-dimensional pixel coordinates of the image and the real-world coordinates.

[0003] Among the current monocular ranging methods, on the one hand, the inverse perspective transformation method is used to calculate the physical coordinates of the object in front through the perspective transformation relationship between the original image and the bird's-eye view corresponding to the original image. However, the fisheye camera usually loses a lot of edge image information when calculating based on this linear transformation method. On the other hand, a monocular depth estimation method based on deep learning is used to detect the depth of the object in the image to obtain the physical coordinates of the object in front, but the network is complex, the amount of training data is large, and the computer resources required are large. Summary of the invention

[0004] Based on this, it is necessary to provide a training method, device, computer equipment, computer-readable storage medium and computer program product for a position prediction model that can accurately and efficiently measure the distance between a robot and a physical object in order to address the above technical problems.

[0005] In a first aspect, the present application provides a method for training a location prediction model, comprising:

[0006] Acquire a target pixel position of a target reference object in a reference image, and a real physical position of a physical reference object determined based on the position of the robot in a physical space, wherein the reference image is an image obtained by the robot photographing the physical reference object, and the target reference object is an image representation of the physical reference object in the reference image;

[0007] Inputting the target pixel position into a position prediction model to be trained for processing to obtain a predicted physical position corresponding to the physical reference object;

[0008] Based on the difference between the predicted physical location and the actual physical location, updating the model parameters of the location prediction model to obtain an updated location prediction model;

[0009] The target pixel position is re-input into the updated position prediction model for processing until a trained position prediction model is obtained, wherein the trained position prediction model is used to obtain the predicted physical position of the physical object to be measured in the target physical space based on the pixel position of the target object to be measured in the target image, so as to obtain the distance between the physical object to be measured and the robot, wherein the target object to be measured is an image representation of the physical object to be measured in the target image.

[0010] In one embodiment, obtaining the real physical position of the physical reference object determined based on the position of the robot in the physical space includes: obtaining the position of the robot in the physical space, and the position of a physical reference point in the physical reference object in the physical space; obtaining the real physical position of the physical reference point in the physical space determined based on the position of the robot according to the positional relationship between the robot and the physical reference point; obtaining the real physical position of each physical reference point in the physical space determined based on the position of the robot according to the positional relationship between the physical reference point and each physical reference point in the physical reference object; and obtaining the real physical position corresponding to the physical reference object according to the real physical positions corresponding to each physical reference point.

[0011] In one of the embodiments, obtaining the target pixel position of the target reference object in the reference image includes: obtaining the target pixel position of the target reference object in the reference image according to the target pixel positions of each target reference point of the target reference object in the reference image, wherein the target reference point is a pixel representation of the physical reference point in the reference image.

[0012] In one of the embodiments, the model parameters of the position prediction model are updated based on the difference between the predicted physical position and the actual physical position, including: in a physical coordinate system determined with the position of the robot as the center, obtaining the predicted physical coordinates of the predicted physical position in the physical coordinate system and the actual physical coordinates of the actual physical position in the physical coordinate system; obtaining the loss of the predicted physical position and the actual physical position based on the component difference between the predicted physical coordinates and the actual physical coordinates; determining the update direction corresponding to the model parameters based on the expected result corresponding to the loss, and updating the model parameters according to the update direction.

[0013] In one embodiment, after re-inputting the target pixel position into the updated position prediction model for processing until the trained position prediction model is obtained, the method further includes: obtaining the target predicted physical position corresponding to each pixel position of the reference image according to the trained position prediction model; determining the pixel coordinates of each pixel position of the reference image in the image coordinate system in the image coordinate system determined according to the reference image; determining the physical coordinates of each target predicted physical position in the physical coordinate system in the physical coordinate system in the physical coordinate system determined with the position of the robot as the center; determining, in each pixel coordinate, the mapping relationship between the components of the pixel coordinate in the image coordinate system and the components of the matching physical coordinate in the physical coordinate system, to obtain a component mapping table corresponding to the pixel coordinates in the image coordinate system and the physical coordinates in the physical coordinate system.

[0014] In one of the embodiments, after obtaining the component mapping table corresponding to the pixel coordinates in the image coordinate system and the physical coordinates in the physical coordinate system, it also includes: determining the target physical area and the target physical coordinates contained in the target physical area in the physical coordinate system determined with the position of the robot as the center; taking the pixel coordinates matching the target physical coordinates as the first pixel coordinates, and retaining the mapping relationship between the first pixel coordinates and the matching target physical coordinates in the component mapping table; taking the pixel coordinates not matching the target physical coordinates as the second pixel coordinates, and eliminating the mapping relationship between the second pixel coordinates and the matching physical coordinates in the component mapping table.

[0015] In one of the embodiments, after obtaining the component mapping table corresponding to the pixel coordinates in the image coordinate system and the physical coordinates in the physical coordinate system, it also includes: obtaining a target image captured by the robot, the target image containing a target object to be measured; obtaining the target pixel coordinates of the target object to be measured in the target image, and obtaining the target predicted physical coordinates corresponding to the physical object to be measured according to the target pixel coordinates in the component mapping table; and obtaining the distance between the physical object to be measured and the robot according to the target predicted physical coordinates of the physical object to be measured.

[0016] In a second aspect, the present application also provides a training device for a location prediction model, comprising:

[0017] an acquisition module, configured to acquire a target pixel position of a target reference object in a reference image, and a real physical position of a physical reference object determined in a physical space based on the position of the robot, wherein the reference image is an image obtained by photographing the physical reference object by the robot, and the target reference object is an image representation of the physical reference object in the reference image;

[0018] A model training module is used to input the target pixel position into a position prediction model to be trained for processing, so as to obtain a predicted physical position corresponding to the physical reference object; based on the difference between the predicted physical position and the actual physical position, the model parameters of the position prediction model are updated to obtain an updated position prediction model; the target pixel position is re-input into the updated position prediction model for processing until a trained position prediction model is obtained, wherein the trained position prediction model is used to obtain a predicted physical position of a physical object to be measured in a target physical space according to the pixel position of the target object to be measured in a target image, so as to obtain a distance between the physical object to be measured and the robot, wherein the target object to be measured is an image representation of the physical object to be measured in the target image.

[0019] In a third aspect, the present application further provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the above steps when executing the computer program.

[0020] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program implements the above steps when executed by a processor.

[0021] In a fifth aspect, the present application also provides a computer program product, including a computer program, which implements the above steps when executed by a processor.

[0022] The training method, apparatus, computer device, computer-readable storage medium and computer program product of the above-mentioned position prediction model first input the target pixel position of the target reference object in the reference image into the position prediction model for processing to obtain the predicted physical position corresponding to the physical reference object, and then update the model parameters of the position prediction model based on the difference between the predicted physical position and the actual physical position of the physical reference object; based on this, the nonlinear transformation relationship between the pixel position in the image and the physical position in the physical space can be adaptively learned through a lightweight position prediction model, that is, in the scenario of monocular ranging through a fisheye camera, the image information can be fully utilized without correcting the distorted area, and the ideal accuracy effect can be achieved without involving a complex network, thereby accurately and efficiently predicting the physical position of the physical object in the robot's operating environment to detect the distance between the robot and the physical object. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0024] Figure 1 A schematic diagram of a flow chart of a method for training a location prediction model in one embodiment;

[0025] Figure 2 is a flowchart of a method for training a location prediction model in another embodiment;

[0026] Figure 3 is a structural block diagram of a training device for a position prediction model in one embodiment;

[0027] Figure 4 is an internal structure diagram of a computer device in one embodiment;

[0028] Figure 5 FIG. 4 is a diagram showing the internal structure of a computer device in another embodiment. DETAILED DESCRIPTION

[0029] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0030] In an exemplary embodiment, a training method for a location prediction model is provided. This embodiment uses the method applied to a terminal as an example. It is understandable that the method can also be applied to a server, and can also be applied to a system including a terminal and a server, and can be implemented through the interaction between the terminal and the server, or can be implemented through the interaction between the terminal and the terminal. Among them, the terminal can be a computer device, such as a robot, a smart wearable device, a self-moving balancing car, etc.; it is understandable that the robot can be a delivery robot (commercial delivery robot, industrial delivery robot, AGV (Automated Guided Vehicle, automatic navigation vehicle) robot), a guide robot, a disinfection robot, a cleaning robot, an inspection robot, and other self-moving robots, which are not specifically limited here. The server can be an independent physical server or a service node in a blockchain system. Each service node in the blockchain system forms a peer-to-peer (P2P) network. The P2P protocol is an application layer protocol running on the Transmission Control Protocol (TCP) protocol. In addition, the server can also be a server cluster composed of multiple physical servers, and can be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms. The robot and the server can be connected via Bluetooth, USB (Universal Serial Bus) or network communication connection methods, which are not limited in this application.

[0031] In an exemplary embodiment, Figure 1 As shown, the method is applied to a robot, which includes steps S102 to S108, wherein:

[0032] Step S102, obtain the target pixel position of the target reference object in the reference image, and the real physical position of the physical reference object determined in the physical space based on the position of the robot, the reference image is an image obtained by the robot photographing the physical reference object, and the target reference object is the image representation of the physical reference object in the reference image.

[0033] Among them, the physical reference object is a physical calibration object that provides calibration and reference functions for the robot in the robot ranging scenario, and the target reference object is a physical reference object represented in the form of an image.

[0034] Among them, the reference image can represent an image obtained by the robot photographing a physical reference object directly in front of the camera lens of the image acquisition module through the image acquisition module; it can also represent an image obtained by the robot photographing a physical reference object in the area in front of the camera lens of the image acquisition module through the image acquisition module, wherein the area in front of the camera lens can be determined by the configuration parameters of the image acquisition module, such as the area that can be captured by the field of view of the camera lens.

[0035] Among them, the target pixel position of the target reference object in the reference image can represent the pixel position of the pattern, edge, contour, key point and other elements corresponding to the target reference object in the reference image; the target pixel position can be represented by the pixel coordinates in the image coordinate system determined by the reference image, for example, taking the pixel point in the upper left corner of the reference image as the origin, and taking the horizontal arrangement direction and vertical arrangement direction of the image pixels as the coordinate axis directions, respectively, to determine the image coordinate system.

[0036] Among them, the real physical position of the physical reference object determined based on the position of the robot in the physical space can represent the actual position of the physical reference object in the physical space reflected based on the relative position of the physical reference object and the robot; the real physical position of the physical reference object can be represented by the physical coordinates in the physical coordinate system determined with the position of the robot as the center, for example, taking the position of the robot as the origin, and the forward direction of the robot and the lateral direction perpendicular to the forward direction as the coordinate axis directions, respectively, to determine the physical coordinate system.

[0037] It can be understood that based on the relative position between the robot and a physical reference object in a specified arrangement direction, the real physical position corresponding to the physical reference object in the robot coordinate system constructed according to the current motion state of the robot can be obtained; further, based on the relative position between the robot and multiple physical reference objects in different arrangement directions, the positional relationship between the real physical positions corresponding to each physical reference object in the global robot coordinate system constructed according to each motion state of the robot can be obtained.

[0038] That is, it is equivalent to obtaining the real physical position corresponding to a physical reference object in a robot coordinate system constructed according to one motion state of the robot; obtaining the real physical position corresponding to another physical reference object in a robot coordinate system constructed according to another motion state of the robot; according to the position relationship and direction relationship of the robot in the above two motion states, the above two robot coordinate systems are merged to obtain a global robot coordinate system; in the global robot coordinate system, the relative positions of physical reference objects in different arrangement directions and the robot can be calculated, and the relative positions of physical reference objects in different arrangement directions can also be calculated.

[0039] Optionally, the photographic lens of the image acquisition module may be a single lens, that is, the image is acquired through a monocular lens; the photographic lens of the image acquisition module may be a fisheye lens, that is, a lens capable of capturing a field of view of nearly 180 degrees or a wider range.

[0040] Step S104: input the target pixel position into the position prediction model to be trained for processing, and obtain the predicted physical position corresponding to the physical reference object.

[0041] The predicted physical position may mean that when the target reference object is mapped to the physical space to obtain the physical reference object, it is predicted that the target pixel position corresponding to the target reference object can be mapped to the physical position corresponding to the obtained physical reference object.

[0042] The position prediction model is used to process the input pixel position to predict the physical position corresponding to the pixel position mapped to the physical space.

[0043] Optionally, the position prediction model may represent a BP (Back Propagation) neural network, and the network may be trained by a back propagation algorithm, i.e., by calculating the gradient of the error, adjusting the weights and biases layer by layer to minimize the error; and the BP neural network model is a two-dimensional neural network, i.e., receiving and processing two-dimensional pixel position data to output two-dimensional physical position data.

[0044] Step S106, based on the difference between the predicted physical location and the actual physical location, the model parameters of the location prediction model are updated to obtain an updated location prediction model.

[0045] Exemplarily, among the target pixel positions corresponding to the target reference object, each target pixel position corresponds to a different predicted physical position and a different real physical position; for one of the target pixel positions, the predicted physical position corresponding to it represents the physical position predicted and mapped by the target pixel position in the physical space, and the real physical position corresponding to it represents the physical position actually mapped by the target pixel position in the physical space. Based on this, according to the difference between the predicted physical position and the real physical position corresponding to each target pixel position, the model parameters of the position prediction model are comprehensively updated.

[0046] Optionally, the difference between the predicted physical location and the actual physical location may be measured by calculation methods such as Mean Squared Error (MSE) and Mean Absolute Error (MAE).

[0047] Step S108, re-input the target pixel position into the updated position prediction model for processing until a trained position prediction model is obtained. The trained position prediction model is used to obtain the predicted physical position of the physical object to be tested in the target physical space based on the pixel position of the target object to be tested in the target image, so as to obtain the distance between the physical object to be tested and the robot. The target object to be tested is an image representation of the physical object to be tested in the target image.

[0048] Optionally, if the difference between the predicted physical position and the actual physical position in each target pixel position is less than or equal to a preset threshold, it indicates that the training of the position prediction model is completed; furthermore, based on the difference between the predicted physical position and the actual physical position corresponding to each target pixel position, the comprehensive difference between the predicted physical position and the actual physical position of all target pixel positions is obtained. If the comprehensive difference is less than or equal to the preset threshold, it indicates that the training of the position prediction model is completed. The above-mentioned preset threshold represents the maximum deviation value allowed between the predicted physical position and the actual physical position, and the maximum deviation value can be determined by the configuration parameters of the image acquisition module, the model structure and model parameters of the position prediction model, the actual requirements for calculation accuracy, and the like.

[0049] Optionally, if the number of iterative trainings of the location prediction model reaches a preset number, it indicates that the training of the location prediction model is completed. The preset number indicates the maximum number of trainings allowed by the model, which may be determined by the model structure and model parameters of the location prediction model, actual requirements for calculation accuracy, and the like.

[0050] Among them, the physical object to be measured can represent the physical object whose distance is to be measured in the robot's forward direction, such as the physical object directly in front of the robot, or the physical object in the area that can be captured by the robot's field of view; the target object to be measured is a physical reference object represented in the form of an image; the target image can represent the image obtained by the robot taking pictures of the physical object to be measured through the image acquisition module. The predicted physical position represents the physical position of the physical object to be measured in the target physical space predicted by the trained position prediction model. The target physical space can represent the physical space where the physical reference object is located during the training of the position prediction model, or it can represent any physical space.

[0051] Optionally, the pixel position of the target object to be tested in the target image is input into a trained position prediction model for processing, and a grounding line parallel to the ground of the target object to be tested in the target image is identified, and the predicted physical position of the physical object to be tested in the target physical space is obtained based on the pixel position corresponding to the grounding line, so as to obtain the distance between the predicted physical position of the physical object to be tested and the position of the robot.

[0052] Optionally, the trained position prediction model is suitable for scenarios where the current robot measures distance to physical objects in the operating environment. The model parameters of the position prediction model may be adjusted according to differences in configuration parameters between different robots. For example, the model parameters may be adjusted according to differences in configuration parameters at levels such as image acquisition and image processing, so that the position prediction model can be applicable to scenarios where different robots measure distance to physical objects in their respective operating environments.

[0053] Optionally, the method can be implemented independently based on a robot, that is, the robot collects reference images, and obtains the target pixel position and the real physical position, and then inputs the target pixel position into a position prediction model configured by the robot for model training, and stores the obtained trained position prediction model in the robot; the method can also be implemented based on a server, that is, the server obtains the reference image collected by the robot, and obtains the target pixel position and the real physical position, and then inputs the target pixel position into the position prediction model configured by the server for model training, and stores the obtained trained position prediction model in the server, or sends it to the robot.

[0054] In the training method of the above-mentioned position prediction model, first, the target pixel position of the target reference object in the reference image is input into the position prediction model for processing to obtain the predicted physical position corresponding to the physical reference object, and then, based on the difference between the predicted physical position of the physical reference object and the actual physical position, the model parameters of the position prediction model are updated; based on this, through a lightweight position prediction model, the nonlinear transformation relationship between the pixel position in the image and the physical position in the physical space can be adaptively learned, that is, in the scenario of monocular ranging through a fisheye camera, the image information can be fully utilized without correcting the distorted area, and the ideal accuracy effect can be achieved without involving a complex network, so as to accurately and efficiently predict the physical position of the physical object in the robot's operating environment to detect the distance between the robot and the physical object.

[0055] In an exemplary embodiment, obtaining the real physical position of the physical reference object in the physical space determined based on the position of the robot includes steps S202 to S208, wherein:

[0056] Step S202, obtaining the position of the robot in the physical space and the position of the physical reference point in the physical reference object in the physical space.

[0057] Step S204, according to the positional relationship between the robot and the physical reference point, obtain the real physical position of the physical reference point in the physical space determined based on the position of the robot.

[0058] Among them, the physical reference points in the physical reference object can represent feature points for calibrating the position of the physical reference object; when the physical reference object is a calibration plate, the physical reference points in the physical reference object can represent special points that can describe the position of the calibration plate, such as points at the center position of the calibration plate and points at the vertex position of the calibration plate.

[0059] Exemplarily, the physical positions of the robot and the physical reference point in the world coordinate system determined according to the global map can be obtained through positioning methods such as GPS, Wi-Fi, and Bluetooth, wherein a global map covering the working area can be constructed based on the working area where the robot is allowed to move in the working environment; then, based on the distance relationship and direction relationship between the physical positions of the robot and the physical reference point in the world coordinate system, the real physical position of the physical reference point in the robot coordinate system determined with the robot's position as the center is obtained, which is equivalent to converting the physical position of the physical reference point in the world coordinate system into a physical position in the robot coordinate system.

[0060] For example, a field survey may be performed in the physical space to measure the distance and direction relationship between the robot and the physical reference point, and obtain the real physical position of the physical reference point in the robot coordinate system determined with the robot's position as the center.

[0061] Step S206, according to the positional relationship between the physical reference point and each physical reference point in the physical reference object, obtain the real physical position of each physical reference point determined based on the position of the robot in the physical space.

[0062] Step S208: obtaining the real physical position corresponding to the physical reference object according to the real physical positions corresponding to the physical reference points.

[0063] Among them, each physical reference point in the physical reference object can represent a feature point in the physical reference object that provides calibration and reference functions for the robot; when the physical reference object is a calibration plate, each physical reference point in the physical reference object can be represented as a corner point corresponding to a mark such as an AprilTag code and an ArUco code in the calibration plate.

[0064] Exemplarily, in the physical reference object, according to the distance relationship and direction relationship between the physical reference point and each physical reference point, and based on the real physical position of the physical reference point in the robot coordinate system, the real physical position of each physical reference point in the robot coordinate system is obtained accordingly, that is, the real physical position corresponding to the physical reference object is obtained.

[0065] In an exemplary embodiment, obtaining a target pixel position of a target reference object in a reference image includes step S210, wherein:

[0066] Step S210, obtaining the target pixel position of the target reference object in the reference image according to the target pixel position of each target reference point of the target reference object in the reference image, where the target reference point is a pixel representation of a physical reference point in the reference image.

[0067] Among them, each target reference point of the target reference object corresponds to each physical reference point of the physical reference object, that is, each target reference point is mapped into the physical space, and physical reference points with actual mapping relationships can be obtained respectively.

[0068] Exemplarily, in the image coordinate system determined according to the reference image, the target pixel position of each target reference point of the target reference object in the image coordinate system can be obtained, that is, the target pixel position of the target reference object in the reference image can be obtained.

[0069] In this embodiment, on the one hand, the position calibration of the physical reference object can be realized according to the physical reference points in the physical reference object, and then the real physical position corresponding to the physical reference object can be accurately determined according to the positional relationship between the physical reference points in the physical reference object and each physical reference point; on the other hand, the target pixel position of the target reference object in the reference image can be accurately determined according to the target pixel position of each target reference point in the target reference object.

[0070] In an exemplary embodiment, based on the difference between the predicted physical location and the actual physical location, the model parameters of the location prediction model are updated, including steps S302 to S306, wherein:

[0071] Step S302 , in a physical coordinate system determined with the position of the robot as the center, obtain predicted physical coordinates of the predicted physical position in the physical coordinate system and real physical coordinates of the real physical position in the physical coordinate system.

[0072] Among them, the physical coordinate system determined with the position of the robot as the center can be represented by a physical coordinate system determined with the position of the robot as the origin, and the robot's forward direction and the lateral direction perpendicular to the forward direction as coordinate axis directions respectively.

[0073] Exemplarily, in the XOY physical coordinate system, the position of the robot is the origin O (0,0), the forward direction of the robot is the X-axis direction, and the direction from the left side to the right side of the robot in the lateral direction perpendicular to the forward direction of the robot is the Y-axis direction; the predicted physical coordinates and the real physical coordinates are expressed in the coordinate form of (x, y), that is, the component in the X-axis direction and the component in the Y-axis direction are expressed.

[0074] Optionally, if the determined physical coordinate system is a three-dimensional physical coordinate system, the vertical direction can be further taken as the Z-axis direction, and the predicted physical coordinates and the actual physical coordinates are expressed in the coordinate form of (x, y, z), that is, the sum of the component in the X-axis direction, the component in the Y-axis direction, and the component in the Z-axis direction are expressed.

[0075] Step S304, obtaining the loss of the predicted physical location and the actual physical location according to the component difference between the predicted physical location and the actual physical location.

[0076] For example, the mean square error can be calculated by calculating each predicted physical coordinate. Respectively correspond to the real physical coordinates The difference between the components is the loss between the predicted physical location and the actual physical location. , calculated as follows:

[0077] (1)

[0078] In formula (1), for each of the n coordinate points corresponding to the physical reference object, the predicted physical coordinate is calculated The corresponding real physical coordinates The difference between the components, that is Represents the i-th physical coordinate The corresponding real physical coordinates The component difference value in the X-axis direction, Represents the i-th physical coordinate The corresponding real physical coordinates Component difference value in the Y-axis direction; traverse n predicted physical coordinates Respectively correspond to the real physical coordinates The component difference between them is used to determine the loss between the predicted physical location and the actual physical location according to the average value corresponding to all component difference values. .

[0079] Step S306, determining the update direction corresponding to the model parameters according to the expected result corresponding to the loss, and updating the model parameters according to the update direction.

[0080] Exemplarily, in the location prediction model, the direction of model training is to make the matching degree between the generated predicted physical location and the real physical location as high as possible, while ensuring that the loss between the predicted physical location and the real physical location is as small as possible, so as to determine the updating direction of the model parameters.

[0081] Optionally, the expected result corresponding to the loss between the predicted physical location and the actual physical location may represent the expected minimum value of the loss between the predicted physical location and the actual physical location. For example, the expected minimum value of the loss obtained by calculating the mean absolute error or the mean square error may be 0.

[0082] Optionally, the expected result may also represent the minimum value that is expected to be achieved when calculating the loss under current computer resource conditions, for example, calculating the loss based on current computer resource conditions such as memory and processor; the expected result may also represent the minimum value that is expected to be achieved when calculating the loss under image quality conditions of a reference image, for example, calculating the loss based on image quality conditions such as image resolution and data volume of the reference image; the expected result may also represent the minimum value that is expected to be achieved in the loss between the predicted physical location and the actual physical location based on dimensions such as computer resource consumption, data processing time, and data processing efficiency, by selecting one or comprehensively considering the above dimensions.

[0083] In this embodiment, in the physical coordinate system determined with the robot's position as the center, the loss between the predicted physical position and the actual physical position is accurately obtained based on the component difference between the predicted physical coordinates and the actual physical coordinates, thereby adaptively determining the update direction of the model parameters of the position prediction model to efficiently update the model parameters according to the update direction.

[0084] In an exemplary embodiment, after the target pixel position is re-input into the updated position prediction model for processing until a trained position prediction model is obtained, the method further includes steps S402 to S408, wherein:

[0085] Step S402, obtaining the target predicted physical position corresponding to each pixel position of the reference image according to the trained position prediction model.

[0086] Among them, the target predicted physical positions corresponding to each pixel position of the reference image can be represented by the physical positions obtained by mapping each pixel position of the reference image predicted by the trained position prediction model to the physical position in the physical space.

[0087] Exemplarily, each pixel position of the reference image is input into the trained position prediction model for processing to obtain the target predicted physical position corresponding to each pixel position, wherein the difference between the target predicted physical position and the corresponding real physical position is less than or equal to the preset threshold in the above embodiment.

[0088] Step S404: In the image coordinate system determined according to the reference image, pixel coordinates of each pixel position of the reference image in the image coordinate system are determined.

[0089] The image coordinate system determined according to the reference image may be represented by an image coordinate system determined by taking a designated pixel point in the reference image as the origin and taking the horizontal arrangement direction and the vertical arrangement direction of the pixels as the coordinate axis directions respectively.

[0090] Exemplarily, in the UOV image coordinate system, the pixel point at the upper left corner of the reference image is taken as the origin O(0,0), the horizontal arrangement direction of the image pixels is the U-axis direction, and the vertical arrangement direction of the image pixels is the V-axis direction; the pixel coordinates are expressed in the coordinate form of (u,v), that is, the component in the U-axis direction and the component in the V-axis direction are expressed.

[0091] Step S406, in a physical coordinate system determined with the position of the robot as the center, determining the physical coordinates of each target predicted physical position in the physical coordinate system.

[0092] Among them, the physical coordinate system determined with the position of the robot as the center can refer to the physical coordinate system determined by taking the position of the robot as the origin, the forward direction of the robot and the lateral direction perpendicular to the forward direction as the coordinate axis directions in the above embodiment, that is: in the XOY physical coordinate system, the position of the robot is taken as the origin O(0,0), the forward direction of the robot is the X-axis direction, and the direction from the left to the right side of the robot is the Y-axis direction; the physical coordinates of each target predicted physical position are expressed in the coordinate form of (x, y), that is, the component in the X-axis direction and the component in the Y-axis direction are expressed.

[0093] Step S408, in each pixel coordinate, determine the mapping relationship between the components of the pixel coordinate in the image coordinate system and the components of the matching physical coordinate in the physical coordinate system, and obtain a component mapping table corresponding to the pixel coordinate in the image coordinate system and the physical coordinate in the physical coordinate system.

[0094] The component mapping table is a corresponding relationship table when converting components in one coordinate system into components in another coordinate system between different coordinate systems.

[0095] Exemplarily, based on the matching relationship between the pixel coordinates in the UOV image coordinate system and the physical coordinates in the XOY physical coordinate system, the mapping relationship between the components of the pixel coordinates in the UOV image coordinate system in the U-axis direction and the components of the physical coordinates in the XOY physical coordinate system in the X-axis direction is determined, as is the mapping relationship between the components of the pixel coordinates in the UOV image coordinate system in the V-axis direction and the components of the physical coordinates in the XOY physical coordinate system in the Y-axis direction.

[0096] Exemplarily, based on the mapping relationship between the components of the pixel coordinates in the UOV image coordinate system in the U-axis direction and the components of the physical coordinates in the X-axis direction in the XOY physical coordinate system, a component mapping table between the components in the X-axis direction and the components in the U-axis direction is obtained; based on the mapping relationship between the components of the pixel coordinates in the V-axis direction in the UOV image coordinate system and the components of the physical coordinates in the Y-axis direction in the XOY physical coordinate system, a component mapping table between the components in the Y-axis direction and the components in the V-axis direction is obtained.

[0097] Optionally, the component mapping table may be stored in a file format such as CSV, JSON, or in a database such as MySQL, so as to efficiently match components in another coordinate system based on components in one coordinate system.

[0098] Optionally, the size of the reference image can be further enlarged to increase the number of pixel coordinates in the reference image, so as to obtain the physical coordinates corresponding to each pixel coordinate of the enlarged reference image, thereby expanding the mapping range between the pixel coordinates in the pixel coordinate system and the physical coordinates in the physical coordinate system; that is, the pixel coordinates in the component mapping table are not limited to the pixel coordinates in the original reference image, and the physical coordinates in the component mapping table are not limited to the physical coordinates corresponding to the pixel coordinates in the original reference image.

[0099] In this embodiment, based on the mapping relationship between the components of the pixel coordinates in the image coordinate system and the components of the matching physical coordinates in the physical coordinate system, a component mapping table corresponding to the pixel coordinates in the image coordinate system and the physical coordinates in the physical coordinate system is obtained completely and accurately, thereby improving the visualization and scalability of the coordinate transformation relationship between the image coordinate system and the physical coordinate system.

[0100] In an exemplary embodiment, after obtaining the component mapping table corresponding to the pixel coordinates in the image coordinate system and the physical coordinates in the physical coordinate system, the method further includes steps S502 to S506, wherein:

[0101] Step S502 , determining a target physical area and each target physical coordinate included in the target physical area in a physical coordinate system determined with the position of the robot as the center.

[0102] The target physical region indicates the physical region to which the physical coordinates whose component mapping relationship with the pixel coordinates in the image coordinate system needs to be retained belong; and the target physical coordinates indicate the physical coordinates within the target physical region in the physical coordinate system.

[0103] Optionally, the target physical area may represent an area that the robot is concerned about in the direction of travel, for example, an area where the robot can detect potential collision risks in the direction of travel, or an area that can be captured by the field of view corresponding to the camera lens in the robot's image acquisition module.

[0104] Step S504: taking the pixel coordinates matching the target physical coordinates as the first pixel coordinates, and retaining the mapping relationship between the first pixel coordinates and the matching target physical coordinates in the component mapping table.

[0105] Step S506: taking the pixel coordinates that do not match the target physical coordinates as the second pixel coordinates, and removing the mapping relationship between the second pixel coordinates and the matching physical coordinates in the component mapping table.

[0106] The first pixel coordinates represent pixel coordinates that are mapped to the target physical coordinates; the second pixel coordinates represent pixel coordinates that are not mapped to the target physical coordinates, that is, the physical coordinates corresponding to the second pixel coordinates are not within the target physical area.

[0107] Exemplarily, in the component mapping table between the components of the XOY physical coordinate system in the X-axis direction and the components of the UOV image coordinate system in the U-axis direction, and in the component mapping table between the components of the XOY physical coordinate system in the Y-axis direction and the components of the UOV image coordinate system in the V-axis direction, the vector mapping relationship between the first pixel coordinate and the matching target physical coordinate is retained, and the vector mapping relationship between the second pixel coordinate and the matching physical coordinate is eliminated.

[0108] Optionally, in the component mapping table, after the vector mapping relationship between the second pixel coordinate and the matching physical coordinate is eliminated, the components of the physical coordinate system and the components of the image coordinate system can be eliminated one by one or at the same time, and an invalid or undefined value, such as NAN, can be added to the position of the eliminated component. A null value, such as Null or None, can also be added.

[0109] In this embodiment, by determining the target physical area in the physical space, the association relationship between each physical coordinate component and the pixel coordinate component in the component mapping table is adaptively retained and eliminated. Therefore, in the actual application scenario, the data content of the component mapping table is adaptively changed according to the actual physical area to be identified to ensure the data applicability of the component mapping table in the current application scenario.

[0110] In an exemplary embodiment, after obtaining the component mapping table corresponding to the pixel coordinates in the image coordinate system and the physical coordinates in the physical coordinate system, the method further includes steps S602 to S606, wherein:

[0111] Step S602, obtaining a target image captured by the robot, where the target image includes a target object to be measured.

[0112] Step S604, obtaining the target pixel coordinates of the target object to be tested in the target image, and obtaining the target predicted physical coordinates corresponding to the physical object to be tested according to the target pixel coordinates matched in the component mapping table.

[0113] Among them, referring to the above-mentioned embodiments, the physical object to be measured may represent a physical object in the direction of the robot's advance and the distance to be measured, such as a physical object directly in front of the robot, or a physical object in the area that can be captured by the robot's field of view; the target object to be measured is a physical reference object represented in the form of an image; the target image may represent an image obtained by the robot taking a picture of the physical object to be measured through an image acquisition module.

[0114] Among them, the target pixel coordinates represent the pixel coordinates corresponding to the target object to be measured in the image coordinate system determined according to the target image; the target predicted physical coordinates represent the physical coordinates corresponding to the physical object to be measured in the physical coordinate system predicted according to the component mapping table.

[0115] Exemplarily, in the image coordinate system, a grounding line of the target object to be measured that is parallel to the ground in the target image is determined, and based on the components of the target pixel coordinates corresponding to the grounding line in the image coordinate system, the components in the physical coordinate system are matched in a component mapping table; based on the matched components in the physical coordinate system, the target predicted physical coordinates corresponding to the target pixel coordinates are combined.

[0116] Step S606, predicting the physical coordinates of the physical object to be measured according to the target, and obtaining the distance between the physical object to be measured and the robot.

[0117] Exemplarily, in a physical coordinate system determined with the position of the robot as the center, the distance between the physical coordinates of the robot and the target predicted physical coordinates of the physical object to be measured, and the direction relationship between the physical object to be measured and the robot are obtained.

[0118] In this embodiment, according to the existing component mapping table, the target pixel coordinates of the target object to be measured can be efficiently and accurately converted into the target predicted physical coordinates corresponding to the physical object to be measured, so as to efficiently and accurately realize the robot's distance measurement of the physical object to be measured and greatly save computer resources.

[0119] In an exemplary embodiment, Figure 2 As shown, the method comprises the following steps:

[0120] Step S702: Determine the real physical coordinates and pixel coordinates corresponding to the corner points of each AprilTag code using a calibration plate including multiple AprilTag codes.

[0121] Specifically, a calibration plate containing multiple AprilTag codes is placed in the front area of ​​the robot, which can be the field of view that can be captured by the camera lens in the image acquisition module of the robot; the robot coordinate system is determined with the position of the robot as the origin (0,0), the robot's forward direction and the lateral direction perpendicular to the forward direction as the coordinate axis directions of the X-axis and the Y-axis respectively; in the robot coordinate system, the physical coordinates corresponding to the center of the calibration plate are determined according to the positional relationship between the center of the calibration plate and the position of the robot; and the real physical coordinates corresponding to the corner points of each AprilTag code are determined according to the positional relationship between the center of the calibration plate and the corner points of each AprilTag code. .

[0122] The robot uses the image acquisition module to capture the calibration plate to obtain a reference image. The image coordinate system is determined by taking the pixel point in the upper left corner of the reference image as the origin (0,0), and taking the horizontal and vertical arrangement directions of the image pixels as the coordinate axis directions of the U axis and the V axis respectively. In the image coordinate system, the pixel coordinates corresponding to the corner points of the AprilTag code of the calibration plate in the reference image are determined. Among them, open source libraries such as OpenCV (Open Source Computer Vision Library) for image processing can be used to identify the corner points of the AprilTag code and detect the pixel coordinates.

[0123] Step S704: input the pixel coordinates corresponding to the corner points of each AprilTag code into the BP neural network for processing, and predict the predicted physical coordinates corresponding to the corner points of each AprilTag code.

[0124] Specifically, a BP neural network is created, and the input and output of the BP neural network are both two-dimensional data; the pixel coordinates corresponding to the corner points of each AprilTag code are Input into the BP neural network for processing, and predict the predicted physical coordinates corresponding to the corner points of each AprilTag code .

[0125] Step S706: Calculate the loss between the predicted physical coordinates and the real physical coordinates corresponding to the corner points of each AprilTag code by calculating the mean square error, and train the BP neural network by minimizing the mean square error.

[0126] Specifically, the predicted physical coordinates corresponding to the corner points of each AprilTag code are calculated by the mean square error. and real physical coordinates The loss between them is calculated to minimize the mean square error as the update direction of the model parameters of the BP neural network to realize the training of the BP neural network; until the mean square error is less than the preset threshold, the training of the BP neural network is completed.

[0127] Step S708: Determine the target predicted physical coordinates corresponding to each pixel coordinate in the reference image, associate the components of the pixel coordinates with the components of the matched target predicted physical coordinates and store them in a component mapping table.

[0128] Specifically, a component mapping table MapX corresponding to the components in the X-axis direction and a component mapping table MapY corresponding to the components in the Y-axis direction are created, the length of the component mapping table MapX matches the width of the reference image, and the length of the component mapping table MapY matches the height of the reference image.

[0129] Traverse each pixel coordinate (u, v) in the reference image and input them into the trained BP neural network for processing respectively, and obtain the target predicted physical coordinates (x, y) corresponding to each pixel coordinate (u, v) in the reference image respectively, and store the mapping relationship between the components of each pixel coordinate (u, v) in the U-axis direction and the components of the matched target predicted physical coordinates (x, y) in the X-axis direction in the component mapping table MapX, and store the mapping relationship between the components of each pixel coordinate (u, v) in the V-axis direction and the components of the matched target predicted physical coordinates (x, y) in the Y-axis direction in the component mapping table MapY.

[0130] Step S710: according to the area that the robot is concerned about in the forward direction, a ROI area is set, and the pixel coordinates corresponding to the physical coordinates that are not in the ROI area are removed from the component mapping table to obtain an updated component mapping table.

[0131] Specifically, according to the area that the robot is concerned about in the forward direction, the ROI area (Region of Interest) is set, for example, a rectangular area is determined by extending 1 meter to the left side of the robot, 1 meter to the right side of the robot, and 2 meters in the front direction of the robot. The pixel coordinates corresponding to the physical coordinates that are not in the ROI area are determined, and the components corresponding to the pixel coordinates are removed from the component mapping table MapX and the component mapping table MapY, and the NAN value is added to the original position to obtain an updated component mapping table.

[0132] Step S712: photographing the physical object to be tested to obtain a target image including the target object to be tested, where the target object to be tested is an image representation of the physical object to be tested.

[0133] Specifically, in a scenario where the distance between the robot and a physical object to be tested needs to be detected, the robot photographs the physical object to be tested through an image acquisition module to obtain a target image containing the target object to be tested, wherein the target object to be tested is an image representation of the physical object to be tested.

[0134] Step S714: Identify a grounding line parallel to the ground of the target object in the target image, and obtain the target predicted physical coordinates corresponding to the target pixel coordinates by matching in a component mapping table according to the components of the target pixel coordinates corresponding to the grounding line.

[0135] Specifically, a grounding line of the target object to be tested that is parallel to the ground in the target image is identified, and according to the components of the target pixel coordinates of the grounding line in the image coordinate system, the target predicted physical coordinates corresponding to the target pixel coordinates are obtained by matching the component mapping table MapX and the component mapping table MapY;

[0136] Step S716: Predict the physical coordinates of the physical object to be measured according to the target, and obtain the distance between the physical object to be measured and the robot.

[0137] Specifically, in the robot coordinate system, the distance between the physical object to be measured and the robot is obtained according to the positional relationship between the target predicted physical coordinates of the physical object to be measured and the physical coordinates of the robot.

[0138] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0139] Based on the same inventive concept, the embodiment of the present application also provides a training device for a position prediction model for implementing the training method for the position prediction model involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in the embodiments of the training device for one or more position prediction models provided below can refer to the limitations of the training method for the position prediction model above, and will not be repeated here.

[0140] In an exemplary embodiment, Figure 3 As shown, a training device for a location prediction model is provided, including: an acquisition module 302 and a model training module 304, wherein:

[0141] An acquisition module 302 is used to acquire a target pixel position of a target reference object in a reference image, and a real physical position of a physical reference object determined based on the position of the robot in a physical space, wherein the reference image is an image obtained by the robot photographing the physical reference object, and the target reference object is an image representation of the physical reference object in the reference image;

[0142] The model training module 304 is used to input the target pixel position into the position prediction model to be trained for processing, so as to obtain the predicted physical position corresponding to the physical reference object; based on the difference between the predicted physical position and the actual physical position, the model parameters of the position prediction model are updated to obtain an updated position prediction model; the target pixel position is re-input into the updated position prediction model for processing until a trained position prediction model is obtained, and the trained position prediction model is used to obtain the predicted physical position of the physical object to be measured in the target physical space according to the pixel position of the target object to be measured in the target image, so as to obtain the distance between the physical object to be measured and the robot, and the target object to be measured is an image representation of the physical object to be measured in the target image.

[0143] In an exemplary embodiment, the acquisition module 302 is also used to: acquire the position of the robot in the physical space, and the position of the physical reference point in the physical reference object in the physical space; obtain the real physical position of the physical reference point in the physical space determined based on the position of the robot according to the positional relationship between the robot and the physical reference point; obtain the real physical position of each physical reference point in the physical space determined based on the position of the robot according to the positional relationship between the physical reference point and each physical reference point in the physical reference object; obtain the real physical position corresponding to the physical reference object according to the real physical positions corresponding to each physical reference point.

[0144] In an exemplary embodiment, the acquisition module 302 is also used to obtain the target pixel position of the target reference object in the reference image according to the target pixel positions of each target reference point of the target reference object in the reference image, where the target reference point is the pixel representation of the physical reference point in the reference image.

[0145] In an exemplary embodiment, the model training module 304 is also used to: obtain the predicted physical coordinates of the predicted physical position in the physical coordinate system and the real physical coordinates of the real physical position in the physical coordinate system in a physical coordinate system determined with the robot's position as the center; obtain the loss of the predicted physical position and the real physical position based on the component difference between the predicted physical coordinates and the real physical coordinates; determine the update direction corresponding to the model parameters based on the expected results corresponding to the loss, and update the model parameters according to the update direction.

[0146] In an exemplary embodiment, the device also includes a component mapping module, which is used to: obtain the target predicted physical position corresponding to each pixel position of the reference image according to the trained position prediction model; determine the pixel coordinates of each pixel position of the reference image in the image coordinate system in the image coordinate system determined according to the reference image; determine the physical coordinates of each target predicted physical position in the physical coordinate system in the physical coordinate system in the physical coordinate system; in each pixel coordinate, determine the mapping relationship between the component of the pixel coordinate in the image coordinate system and the component of the matching physical coordinate in the physical coordinate system, and obtain a component mapping table corresponding to the pixel coordinate in the image coordinate system and the physical coordinate in the physical coordinate system.

[0147] In an exemplary embodiment, the component mapping module is also used to: determine the target physical area and the target physical coordinates contained in the target physical area in a physical coordinate system determined with the robot's position as the center; use the pixel coordinates that match the target physical coordinates as the first pixel coordinates, and retain the mapping relationship between the first pixel coordinates and the matching target physical coordinates in the component mapping table; use the pixel coordinates that do not match the target physical coordinates as the second pixel coordinates, and eliminate the mapping relationship between the second pixel coordinates and the matching physical coordinates in the component mapping table.

[0148] In an exemplary embodiment, the component mapping module is also used to: obtain a target image captured by the robot, the target image contains a target object to be measured; obtain the target pixel coordinates of the target object to be measured in the target image, and obtain the target predicted physical coordinates corresponding to the physical object to be measured according to the target pixel coordinates in the component mapping table; obtain the distance between the physical object to be measured and the robot according to the target predicted physical coordinates of the physical object to be measured.

[0149] Each module in the training device of the above-mentioned position prediction model can be implemented in whole or in part by software, hardware and a combination thereof. Each of the above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory in the computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.

[0150] In an exemplary embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as shown in FIG. Figure 4 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data involved in the training process and application process of the location prediction model. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a training method for a location prediction model is implemented.

[0151] In an exemplary embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as shown in FIG. Figure 5As shown. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit and an input device. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface, the display unit and the input device are connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a mobile cellular network, near field communication (NFC) or other technologies. When the computer program is executed by the processor, a training method for a position prediction model is implemented. The display unit of the computer device is used to form a visually visible picture, which can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the computer device shell, or an external keyboard, touchpad or mouse.

[0152] Those skilled in the art will understand that Figure 4 or Figure 5 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0153] In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps of the above-mentioned location prediction model training method when executing the computer program.

[0154] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the training method of the above-mentioned position prediction model are implemented.

[0155] In one embodiment, a computer program product is provided, including a computer program, which implements the steps of the above-mentioned training method of the position prediction model when executed by a processor.

[0156] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0157] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0158] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A method for training a position prediction model, characterized in that: The method comprises: Acquire a target pixel position of a target reference object in a reference image, and a real physical position of a physical reference object determined based on the position of the robot in a physical space, wherein the reference image is an image obtained by the robot photographing the physical reference object, and the target reference object is an image representation of the physical reference object in the reference image; Inputting the target pixel position into a position prediction model to be trained for processing to obtain a predicted physical position corresponding to the physical reference object; Based on the difference between the predicted physical location and the actual physical location, updating the model parameters of the location prediction model to obtain an updated location prediction model; The target pixel position is re-input into the updated position prediction model for processing until a trained position prediction model is obtained, wherein the trained position prediction model is used to obtain the predicted physical position of the physical object to be measured in the target physical space based on the pixel position of the target object to be measured in the target image, so as to obtain the distance between the physical object to be measured and the robot, wherein the target object to be measured is an image representation of the physical object to be measured in the target image.

2. The method according to claim 1, characterized in that The obtaining of the real physical position of the physical reference object in the physical space determined based on the position of the robot comprises: Acquire the position of the robot in the physical space and the position of the physical reference point in the physical reference object in the physical space; According to the positional relationship between the robot and the physical reference point, a real physical position of the physical reference point in the physical space determined based on the position of the robot is obtained; According to the positional relationship between the physical reference point and each physical reference point in the physical reference object, obtain the real physical position of each physical reference point determined based on the position of the robot in the physical space; According to the real physical positions corresponding to the various physical reference points, the real physical position corresponding to the physical reference object is obtained.

3. The method according to claim 1, characterized in that The step of obtaining a target pixel position of a target reference object in a reference image includes: The target pixel position of the target reference object in the reference image is obtained according to the target pixel position of each target reference point of the target reference object in the reference image, and the target reference point is the pixel representation of the physical reference point in the reference image.

4. The method according to claim 1, characterized in that The updating of the model parameters of the location prediction model based on the difference between the predicted physical location and the actual physical location includes: In a physical coordinate system determined with the position of the robot as the center, obtaining predicted physical coordinates of the predicted physical position in the physical coordinate system and real physical coordinates of the real physical position in the physical coordinate system; Obtaining a loss between the predicted physical position and the real physical position according to a difference between components of the predicted physical coordinates and the real physical coordinates; According to the expected result corresponding to the loss, the update direction corresponding to the model parameters is determined, and the model parameters are updated according to the update direction.

5. The method according to claim 1, characterized in that The step of re-inputting the target pixel position into the updated position prediction model for processing until a trained position prediction model is obtained further includes: Obtaining, according to the trained position prediction model, the predicted physical positions of the targets corresponding to the respective pixel positions of the reference image; In an image coordinate system determined according to the reference image, determining pixel coordinates of each pixel position of the reference image in the image coordinate system; In a physical coordinate system determined with the position of the robot as the center, determining the physical coordinates of each target predicted physical position in the physical coordinate system; In each pixel coordinate, the mapping relationship between the components of the pixel coordinate in the image coordinate system and the components of the matching physical coordinate in the physical coordinate system is determined to obtain a component mapping table corresponding to the pixel coordinate in the image coordinate system and the physical coordinate in the physical coordinate system.

6. The method according to claim 5, characterized in that After obtaining the component mapping table corresponding to the pixel coordinates in the image coordinate system and the physical coordinates in the physical coordinate system, the method further includes: Determine a target physical area and each target physical coordinate included in the target physical area in a physical coordinate system determined with the position of the robot as the center; Taking the pixel coordinates matching the target physical coordinates as the first pixel coordinates, and retaining the mapping relationship between the first pixel coordinates and the matching target physical coordinates in the component mapping table; The pixel coordinates that do not match the target physical coordinates are used as second pixel coordinates, and the mapping relationship between the second pixel coordinates and the matching physical coordinates in the component mapping table is eliminated.

7. The method according to claim 5, characterized in that After obtaining the component mapping table corresponding to the pixel coordinates in the image coordinate system and the physical coordinates in the physical coordinate system, the method further includes: Acquire a target image acquired by the robot, wherein the target image includes a target object to be measured; Obtaining target pixel coordinates of the target object to be measured in the target image, and obtaining target predicted physical coordinates corresponding to the physical object to be measured by matching the target pixel coordinates in the component mapping table; The distance between the physical object to be measured and the robot is obtained according to the target predicted physical coordinates of the physical object to be measured.

8. A training device for a position prediction model, characterized in that: The device comprises: an acquisition module, configured to acquire a target pixel position of a target reference object in a reference image, and a real physical position of a physical reference object determined in a physical space based on the position of the robot, wherein the reference image is an image obtained by photographing the physical reference object by the robot, and the target reference object is an image representation of the physical reference object in the reference image; A model training module is used to input the target pixel position into a position prediction model to be trained for processing, so as to obtain a predicted physical position corresponding to the physical reference object; based on the difference between the predicted physical position and the actual physical position, the model parameters of the position prediction model are updated to obtain an updated position prediction model; the target pixel position is re-input into the updated position prediction model for processing until a trained position prediction model is obtained, wherein the trained position prediction model is used to obtain a predicted physical position of a physical object to be measured in a target physical space according to the pixel position of the target object to be measured in a target image, so as to obtain a distance between the physical object to be measured and the robot, wherein the target object to be measured is an image representation of the physical object to be measured in the target image.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images