Self-moving equipment system for parameter configuration of camera component

Abstract

The invention discloses a self-moving equipment system for camera component parameter configuration. The system comprises a self-moving device and a charging pile, and the charging pile is used for charging the self-moving device. The self-moving device includes: a body; the walking wheel assembly is used for supporting the machine body; the camera assembly is mounted on the machine body and is used for collecting images around the self-moving equipment; the controller is in communication connection with the camera assembly and pre-stores three-dimensional feature points pre-calibrated on the charging pile based on preset feature information; the controller is configured to acquire a target image corresponding to the charging pile acquired by the camera assembly, and perform feature point extraction on the target image based on preset feature information to obtain two-dimensional feature points; and performing parameter calibration on the camera assembly based on the corresponding relationship between the two-dimensional feature points and the three-dimensional feature points. According to the method and the device, the camera assembly parameters can be quickly and accurately calibrated based on the corresponding relationship between the three-dimensional feature points of the charging pile and the two-dimensional feature points of the charging pile image under the same feature information.

Description

Technical Field

[0001] This application relates to the technical field of power tools, and particularly to a self - moving device system for parameter configuration of a camera component. Background Art

[0002] With the rapid development of power tool technology, self - moving devices have gradually attracted attention and are widely used in various operation scenarios, such as self - moving lawn mowers for lawn mowing operations. An image acquisition device (such as a camera or a webcam, etc.) is usually deployed in a self - moving device, and the positioning and movement of the self - moving device are realized through the image acquisition device.

[0003] However, when the image acquisition device in the self - moving device works for a long time, structural problems such as lens or circuit board aging may occur, resulting in deviations between the relevant parameters of the image acquisition device and the actual situation, that is, the parameters are inaccurate, thus affecting the working accuracy of the image acquisition device. Therefore, to ensure the working accuracy of the image acquisition device, it is necessary to accurately calibrate the relevant parameters of the image acquisition device.

[0004] This section provides background information related to this application, and these background information are not necessarily prior art. Summary of the Invention

[0005] An object of this application is to solve or at least mitigate some or all of the above problems. For this reason, an object of this application is to provide a self - moving device system for parameter configuration of a camera component.

[0006] To achieve the above object, this application adopts the following technical solutions:

[0007] A self - moving device system for parameter configuration of a camera component includes a self - moving device and a charging pile for charging the self - moving device; the self - moving device includes: a body; a walking wheel assembly for supporting the body; a camera component installed on the body for collecting images around the self - moving device; a controller communicatively connected to the camera component, and the controller prestores three - dimensional feature points on the charging pile; wherein, the three - dimensional feature points are pre - calibrated based on preset feature information; the controller is configured to: obtain a target image corresponding to the charging pile collected by the camera component, and extract feature points from the target image based on the preset feature information to obtain two - dimensional feature points; and calibrate the parameters of the camera component based on the correspondence between the two - dimensional feature points and the three - dimensional feature points.

[0008] In some embodiments, the camera component includes a binocular camera, and the binocular camera includes a left camera and a right camera.

[0009] In some embodiments, the target image includes a first image and a second image. The first image is the image captured by the left camera, and the second image is the image captured by the right camera.

[0010] In some embodiments, extracting two-dimensional feature points from the target image based on the preset feature information includes: extracting feature points from the first image based on the preset feature information to obtain first-image two-dimensional feature points corresponding to the first image; and extracting feature points from the second image based on the preset feature information to obtain second-image two-dimensional feature points corresponding to the second image.

[0011] In some embodiments, calibrating the parameters of the camera assembly based on the correspondence between the two-dimensional feature points and the three-dimensional feature points includes: determining first internal parameters of the left camera based on the correspondence between the first-image two-dimensional feature points and the three-dimensional feature points; determining second internal parameters of the right camera based on the correspondence between the second-image two-dimensional feature points and the three-dimensional feature points.

[0012] In some embodiments, the internal parameters include focal length and optical center.

[0013] In some embodiments, calibrating the parameters of the camera assembly based on the correspondence between the two-dimensional feature points and the three-dimensional feature points includes: determining first external parameters of the left camera based on the correspondence between the first-image two-dimensional feature points and the three-dimensional feature points and the first internal parameters; wherein the first external parameters are used to represent the relative pose information between the left camera and the charging pile; determining second external parameters of the right camera based on the correspondence between the second-image two-dimensional feature points and the three-dimensional feature points and the second internal parameters; wherein the second external parameters are used to represent the relative pose information between the right camera and the charging pile; determining third external parameters of the binocular camera according to the first external parameters and the second external parameters; wherein the third external parameters are used to represent the relative pose information between the left camera and the right camera.

[0014] In some embodiments, the preset feature information includes structural information and additional flag information.

[0015] In some embodiments, the structural information includes at least one of the following: edge information, corner information, and texture information.

[0016] In some embodiments, the additional flag information includes preset pattern information, and the preset pattern information includes two-dimensional code information.

[0017] In some embodiments, the self-mobile device system further includes an infrared laser, and the camera assembly further includes a low-light camera.

[0018] In some embodiments, the controller is further configured to: determine the working scenario of the self - moving device, where the working scenario is day or night; and determine the working mode of the camera assembly in the self - moving device according to the working scenario.

[0019] In some embodiments, determining the working scenario of the self - moving device includes: determining the working scenario of the self - moving device based on the working time of the self - moving device or the ambient brightness where the self - moving device is located.

[0020] In some embodiments, determining the working mode of the camera assembly in the self - moving device according to the working scenario includes: if the working scenario is day, determining the working mode of the camera assembly as a low - light camera and a binocular camera in a first state, or only the binocular camera in the first state; where the first state is that the infrared laser is in the off state; if the working scenario is night, determining the working mode of the camera assembly as a low - light camera and a binocular camera in a second state; where the second state is that the infrared laser is in the on state.

[0021] The advantages of the present application are as follows:

[0022] A self - moving device system for camera assembly parameter configuration provided by the present application includes a self - moving device and a charging pile. The charging pile is used to charge the self - moving device. The self - moving device includes: a body; a walking wheel assembly for supporting the body; a camera assembly installed on the body for collecting images around the self - moving device; a controller communicatively connected to the camera assembly. The controller pre - stores three - dimensional feature points on the charging pile; where the three - dimensional feature points are pre - calibrated based on preset feature information. The controller is configured to: obtain a target image corresponding to the charging pile collected by the camera assembly, and perform feature point extraction on the target image based on the preset feature information to obtain two - dimensional feature points; and perform parameter calibration on the camera assembly based on the correspondence between the two - dimensional feature points and the three - dimensional feature points. The self - moving device system provided by the present application can, based on the correspondence between the three - dimensional feature points of the charging pile and the two - dimensional feature points of the charging pile image under the same feature information, achieve rapid and accurate calibration of the parameters of the camera assembly, thereby solving the problem of inaccurate calibration of the parameters of the camera assembly caused by structural factors such as aging of the lens or circuit board of the camera assembly, and helping to improve the working accuracy of the camera assembly. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 is a schematic diagram of a self - moving device system provided by the present application;

[0024] Figure 2 is a schematic diagram of a self - moving lawn mower provided by the present application;

[0025] Figure 3 is a schematic diagram of a charging pile provided by this application;

[0026] Figure 4 is a flowchart of parameter configuration (parameter calibration) of a camera component provided by this application;

[0027] Figure 5 is a schematic diagram of a pinhole imaging model provided by this application;

[0028] Figure 6 is a kind of Figure 5 schematic diagram of a simplified model provided by this application;

[0029] Figure 7 is another flowchart of parameter configuration (working mode determination) of a camera component provided by this application;

[0030] Figure 8 is a schematic structural diagram of a controller provided by this application;

[0031] Figure 9 is a schematic structural diagram of another controller provided by this application.

[0032] Reference numerals:

[0033] 110, fuselage; 120, traveling wheel assembly; 130, camera component; 140, controller. Detailed implementation manners

[0034] Before explaining any implementation manners of this application in detail, it should be understood that this application is not limited to the structural details and component arrangements described in the following description or shown in the above drawings.

[0035] In this application, the terms "include", "comprise", "have" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements but also includes other elements not expressly listed, or further includes elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "including one..." does not exclude the existence of additional identical elements in the process, method, article or device including that element.

[0036] In this application, the term "and / or" is an associative relationship describing associated objects, indicating that three relationships may exist. For example, A and / or B may represent: A exists alone, A and B exist simultaneously, and B exists alone. In addition, the character " / " in this application generally represents an "and / or" relationship between the associated objects before and after.

[0037] In this application, the terms "connected", "joined", "coupled", and "mounted" may be direct connections, joins, couplings, or mountings, or may be indirect connections, joins, couplings, or mountings. For example, a direct connection means that two parts or components are connected together without an intermediate member, and an indirect connection means that two parts or components are each connected to at least one intermediate member, and these two parts or components are connected through the intermediate member. In addition, "connected" and "coupled" are not limited to physical or mechanical connections or couplings, and may include electrical connections or couplings.

[0038] In this application, those of ordinary skill in the art will understand that relative terms used in connection with a quantity or condition (e.g., "about", "approximately", "substantially", etc.) are intended to include the recited value and have the meaning indicated by the context. For example, such relative terms include at least the degree of error associated with the measurement of a particular value, tolerances resulting from manufacturing, assembly, use in relation to a particular value, etc. Such terms should also be considered to disclose a range defined by the absolute values of two endpoints. Relative terms may refer to a plus or minus a certain percentage (e.g., 1%, 5%, 10% or more) of the indicated value. Numerical values without the use of relative terms should also be disclosed as specific values having tolerances. In addition, "substantially" in expressing a relative angular positional relationship (e.g., substantially parallel, substantially perpendicular) may refer to a plus or minus a certain number of degrees (e.g., 1 degree, 5 degrees, 10 degrees or more) from the indicated angle.

[0039] In this application, those of ordinary skill in the art will understand that the functions performed by a component may be performed by one component, multiple components, one part, or multiple parts. Similarly, the functions performed by a part may also be performed by one part, one component, or a combination of multiple parts.

[0040] In this application, the directional terms such as "upper", "lower", "left", "right", "front", "rear", etc. are described based on the orientation and positional relationship shown in the drawings, and should not be construed as a limitation on the embodiments of this application. In addition, in the context, it should also be understood that when it is mentioned that one element is connected "above" or "below" another element, it can not only be directly connected "above" or "below" another element, but also be indirectly connected "above" or "below" another element through an intermediate element. It should also be understood that the directional terms such as upper side, lower side, left side, right side, front side, rear side, etc. not only represent the positive direction, but can also be understood as the side direction. For example, the lower side may include directly below, lower left, lower right, front lower, and rear lower, etc.

[0041] It should be noted that the terms "first", "second", etc. in the description, claims and the above-mentioned drawings of this application are used to distinguish similar objects, and do not necessarily have to be used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so that the embodiments of the present application described here can be implemented in an order other than those illustrated or described here. In addition, the term "comprising" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device comprising a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0042] In this application, the terms "controller", "processor", "central processing unit", "CPU", "MCU" can be interchanged. When using the units "controller", "processor", "central processing unit", "CPU", or "MCU" to perform specific functions, unless otherwise specified, these functions can be performed by a single one of the above units or multiple ones of the above units.

[0043] In this application, the terms "device", "module" or "unit" can be implemented in the form of hardware or software in order to achieve specific functions.

[0044] In this application, the terms "calculate", "judge", "control", "determine", "identify", etc. refer to the operations and processes of a computer system or similar electronic computing devices (such as a controller, a processor, etc.).

[0045] The technical solutions proposed in this application will be further described in detail below in conjunction with the accompanying drawings and specific embodiments.

[0046] See Figure 1 , the self-moving device system in this embodiment includes a self-moving device and a charging pile. Among them, the self-moving device can refer to an intelligent device that can achieve automatic positioning and movement. Hereinafter, a self-moving lawn mower will be used as an example for illustration. See Figure 2 , the self-moving lawn mower includes a body (110), a walking wheel assembly (120), a camera assembly (130) and a controller (140). Among them, the walking wheel assembly (120) can be used to support the body (110); the camera assembly (130) is installed on the body (110) and can be used to collect images around the self-moving lawn mower; the controller (140) is located inside the self-moving lawn mower and is communicatively connected (such as electrically connected) to the camera assembly (130). Exemplarily, the camera assembly can be a camera or a webcam. See Figure 3 , the charging pile can be understood as an external charging device with a fixed size, which can be used to charge the self-moving device (such as a self-moving lawn mower).

[0047] It should be noted that the three-dimensional feature points of the charging pile are pre-stored in the controller of the self-mobile device. For example, they are stored in a register or a memory. Among them, the three-dimensional feature points are pre-calibrated based on preset feature information. The preset feature information may refer to the feature information preset according to actual application requirements. Optionally, the preset feature information includes structure information and additional flag information. Optionally, the structure information includes at least one of the following: edge information, corner point information, and texture information. Optionally, the additional flag information includes preset pattern information, where the preset pattern information includes two-dimensional code information, bar code information, etc. Exemplarily, the three-dimensional feature points can be characterized by world coordinate information. Specifically, a world coordinate system can be established in advance, and the charging pile with three-dimensional feature points pre-calibrated based on the preset feature information is projected into the world coordinate system, so that the spatial coordinate information corresponding to each three-dimensional feature point can be obtained, and then the spatial coordinate information is stored in the controller of the self-mobile device.

[0048] See Figure 4 , the controller of the self-mobile device is configured to execute the following steps A1 - A2:

[0049] A1, obtain the target image corresponding to the charging pile collected by the camera assembly, and perform feature point extraction on the target image based on the preset feature information to obtain two-dimensional feature points.

[0050] Specifically, when the self-mobile device walks near the charging pile, the charging pile can be imaged by the camera assembly in the self-mobile device to obtain the target image corresponding to the charging pile. Then, feature point detection and extraction are performed on the target image according to the preset feature information to obtain the two-dimensional feature points in the target image. Among them, the number of two-dimensional feature points is the same as that of the three-dimensional feature points pre-stored in the controller, and the two-dimensional feature points can be characterized by two-dimensional coordinate information. Specifically, the target image can be used as a reference to establish a pixel coordinate system. For example, with the upper left corner of the target image as the center, the horizontal pixel direction (i.e., the direction where the width is located) and the vertical pixel direction (i.e., the direction where the height is located) of the target image are used as the horizontal axis and the vertical axis respectively to establish a pixel coordinate system, so that the two-dimensional coordinate information of each two-dimensional feature point in the pixel coordinate system can be obtained.

[0051] It should be noted that in this embodiment, the feature point detection method is not specifically limited, and can be flexibly selected according to actual application requirements. Exemplarily, for edge information detection, a first-order detection algorithm (such as Roberts operator, Prewitt operator, Sobel operator, Canny operator, etc.) can be used, or a second-order detection algorithm (such as Laplacian operator, etc.) can be used; for corner detection, algorithms such as SIFT (Scale Invariant Feature Transform), Harris, SURF, or FAST (features from accelerated segment test) can be used; for texture detection, methods such as GLCM (Gray-level co-occurrence matrix) or LBP (Local Binary Pattern) can be used.

[0052] A2. Based on the correspondence between the two-dimensional feature points and the three-dimensional feature points, calibrate the parameters of the camera assembly.

[0053] In this embodiment, since both the two-dimensional feature points and the three-dimensional feature points are feature points obtained based on preset feature information, the difference is only in the dimension of the feature points, and the difference in the dimension of the feature points is exactly caused by the shooting of the camera assembly, and the parameters of the camera assembly play a key role. Therefore, the parameters of the camera assembly can be calibrated based on the correspondence between the two-dimensional feature points and the three-dimensional feature points under the same preset feature information. Among them, the parameters can include internal parameters. Optionally, the internal parameters include focal length and optical center.

[0054] See Figure 5 , regarding the camera (camera assembly) as a pinhole, the correspondence between the two-dimensional feature points and the three-dimensional feature points can be described by the pinhole imaging model. Specifically, the three-dimensional feature point P in the world coordinate system is projected onto the physical imaging plane through the optical center O of the camera to become the corresponding two-dimensional feature point P'. Among them, O-x-y-z is the camera coordinate system, the z-axis points to the front of the camera, the x-axis is to the right, the y-axis is downward, and the distance between the physical imaging plane and the optical center (i.e., the length of OO') is the focal length f of the camera.

[0055] Assume that the coordinates of point P in the world coordinate system are (X, Y, Z), and the coordinates of point P' in the O'-x'-y' coordinate system are (X', Y'), then the Figure 5 pinhole imaging model in can be simplified to Figure 6 similar triangles in. According to Figure 6 the following relationship can be obtained: Furthermore, the coordinate relationship between P and P' can be obtained as The coordinates of point P' in the O'-x'-y' coordinate system can be obtained as follows

[0056] Furthermore, it is also necessary to convert point P' on the physical imaging plane to the pixel plane, that is, to convert point P' from the O'-x'-y' coordinate system to the pixel coordinate system (o'-u-v). Among them, the origin o' is located at the upper left corner of the image, the u-axis is parallel to the x-axis to the right, and the v-axis is parallel to the y-axis downward. Assume that the coordinates of the corresponding point P'' of P' in the o'-u-v coordinate system are (U, V), and the coordinates of point P'' are scaled by α times on the u-axis and β times on the v-axis. At the same time, the origin is translated by [c x , c y , then the coordinate relationship between P' and P'' can be obtained as Substituting the coordinate relationship between P and P' into the above formula, we can get The corresponding matrix form is Among them, f x and f y are in the unit of pixels, and K is the internal parameter matrix of the camera. Thus, the internal parameters of the camera component can be calibrated based on the corresponding relationship between the two-dimensional feature points and the three-dimensional feature points under the same preset feature information.

[0057] In some embodiments, optionally, the camera component includes a binocular camera, where the binocular camera includes a left camera and a right camera. Correspondingly, the target image includes a first image and a second image, where the first image is a charging pile image collected by the left camera, and the second image is a charging pile image collected by the right camera. In this case, the extraction of the feature points from the target image based on the preset feature information to obtain the two-dimensional feature points may specifically include the following steps B1-B2:

[0058] B1. Extract feature points from the first image based on the preset feature information to obtain the first-image two-dimensional feature points corresponding to the first image.

[0059] Among them, the first-image two-dimensional feature points may refer to the two-dimensional feature points corresponding to the first image obtained after extracting the feature points from the first image based on the preset feature information. Among them, the feature point extraction method may refer to the relevant description in step A1 above, and will not be elaborated here.

[0060] B2. Extract feature points from the second image based on the preset feature information to obtain the second-image two-dimensional feature points corresponding to the second image.

[0061] Among them, the second-image two-dimensional feature points may refer to the two-dimensional feature points corresponding to the second image obtained after extracting the feature points from the second image based on the preset feature information. Among them, the feature point extraction method may refer to the relevant description in step A1 above, and will not be elaborated here.

[0062] It should be noted that the binocular camera can operate in grayscale mode or color mode (such as RGB). When the binocular camera operates in grayscale mode, the target images it captures (including the first image and the second image) are grayscale images; when the binocular camera operates in RGB mode, the target images it captures (including the first image and the second image) are RGB images. Further, in order to improve the effect of feature point extraction of the target image, if the target images (including the first image and the second image) captured by the binocular camera are RGB images, it is necessary to first convert the RGB images into grayscale images, and then perform feature point extraction on the grayscale images based on the preset feature information.

[0063] In some embodiments, optionally, step A2 may specifically include the following steps C1-C2:

[0064] C1, determine the first internal parameter of the left camera based on the correspondence between the two-dimensional feature points and three-dimensional feature points of the first image.

[0065] Among them, the first internal parameter may refer to the internal parameter corresponding to the left camera, and the determination method of the first internal parameter may refer to the relevant description in step A2 above, and will not be elaborated here.

[0066] C2, determine the second internal parameter of the right camera based on the correspondence between the two-dimensional feature points and three-dimensional feature points of the second image.

[0067] Among them, the second internal parameter may refer to the internal parameter corresponding to the right camera, and the determination method of the second internal parameter may refer to the relevant description in step A2 above, and will not be elaborated here.

[0068] In some embodiments, optionally, step A2 may specifically further include the following steps D1-D3:

[0069] D1, determine the first external parameter of the left camera based on the correspondence between the two-dimensional feature points and three-dimensional feature points of the first image and the first internal parameter; among them, the first external parameter is used to represent the relative pose information between the left camera and the charging pile.

[0070] Exemplarily, the first external parameter of the left camera can be determined based on the least squares method or the PnP (Perspective-n-Point) algorithm, etc. Among them, the PnP algorithm represents the external parameter based on the rotation matrix and the translation vector. Taking the PnP algorithm as an example, the first external parameter of the left camera can be determined by the following formula:

[0071]

[0072] Among them, p represents the coordinates of a point in the pixel coordinate system, P C represents the coordinates of a point in the camera coordinate system, PW represents the coordinates of a point in the world coordinate system, ω represents the depth of the point, and K represents the internal parameter matrix of the left camera. RCW and are used to characterize the pose transformation from the world coordinate system to the camera coordinate system (i.e., the first extrinsic parameter). Among them, R CW represents the rotation matrix from the world coordinate system to the camera coordinate system (transforming the representation of the same vector in the world coordinate system into the representation in the camera coordinate system), represents the corresponding translation vector (i.e., the representation of the vector pointing from the origin of the camera coordinate system to the origin of the world coordinate system in the camera coordinate system).

[0073] Specifically, the solution process of the first extrinsic parameter of the left camera can be described as the following problem: Given that the coordinates of n three-dimensional feature points in the world coordinate system are and the corresponding two-dimensional feature points of these n three-dimensional feature points in the pixel coordinate system are p1, p2,..., p n , and given that the internal parameter matrix of the left camera is K, solve the pose R of the camera coordinate system relative to the world coordinate system CW and Exemplarily, this problem can be solved based on methods such as DLT (Direct Linear Transformation), P3P, EPnP, or BA (Bundle Adjustment). For specific implementation processes, reference can be made to the relevant content in the prior art, and details will not be elaborated here.

[0074] D2, based on the correspondence between the two-dimensional feature points of the second image and the three-dimensional feature points and the second internal parameter, determine the second extrinsic parameter of the right camera; among them, the second extrinsic parameter is used to characterize the relative pose information between the right camera and the charging pile.

[0075] Among them, the determination method of the second extrinsic parameter can refer to the relevant description in step D1 above, and details will not be elaborated here.

[0076] D3, determine the third extrinsic parameter of the binocular camera according to the first extrinsic parameter and the second extrinsic parameter; among them, the third extrinsic parameter is used to characterize the relative pose information between the left camera and the right camera.

[0077] It can be understood that since the first extrinsic parameter and the second extrinsic parameter respectively represent the relative pose information between the left camera and the right camera and the charging pile, therefore, given the first extrinsic parameter and the second extrinsic parameter, the relative pose information between the left camera and the right camera (i.e., the third extrinsic parameter of the binocular camera) can be quickly determined.

[0078] In some embodiments, optionally, the self-moving device system further includes an infrared laser, and the camera assembly further includes a low-light camera.

[0079] Exemplarily, the low-light camera can be an RGB camera. Further, the low-light camera can be set as a hypersensitive camera with a minimum illumination of less than or equal to 0.01 Lux, which helps to improve the image quality when shooting in a low-light environment at night. Specifically, the low-light camera can work during the day and at night. When the low-light camera works during the day, it can be used for AI semantic perception and positioning; since the color effect at night is not as good as that during the day, when the low-light camera works at night, it cannot be used for AI semantic perception and is only used for positioning, which can specifically include visual positioning of charging piles (such as positioning the target or the position relative to the target) and visual SLAM (such as positioning itself or the absolute position), etc. Further, the infrared laser can also be a near-infrared laser projector.

[0080] See Figure 7 , the controller is further configured to perform the following steps E1 - E2:

[0081] E1, determine the working scenario of the self-mobile device, where the working scenario is day or night.

[0082] In some embodiments, optionally, step E1 can specifically include the following steps: Based on the working time of the self-mobile device or the ambient brightness where the self-mobile device is located, determine the working scenario of the self-mobile device.

[0083] Specifically, the first mapping relationship between the working time of the self-mobile device and the working scenario can be preset according to the actual application scenario, and the second mapping relationship between the ambient brightness where the self-mobile device is located and the working scenario can be preset. Exemplarily, the first mapping relationship can be set as: [7:00 - 19:00] is day; [19:00 - 7:00] is night. The second mapping relationship can be set as the ambient brightness is greater than the preset brightness threshold for day; the ambient brightness is less than or equal to the preset brightness threshold for night. Among them, the preset brightness threshold can refer to the ambient brightness reference value preset according to the actual application requirements.

[0084] Based on the preset first mapping relationship, the current working time of the self-mobile device can be determined first, and the first mapping relationship can be searched according to the current working time to determine the working scenario that matches the current working time as the current working scenario of the self-mobile device. Based on the preset second mapping relationship, the current ambient brightness where the self-mobile device is located can be determined first, and the second mapping relationship can be searched according to the current ambient brightness to determine the working scenario that matches the current ambient brightness as the current working scenario of the self-mobile device. Based on the preset first mapping relationship and the second mapping relationship, one of the methods can be selected to determine the current working scenario of the self-mobile device.

[0085] E2, determine the working mode of the camera component in the self-mobile device according to the working scenario.

[0086] After determining the working scenario of the self - moving device, the working mode of the camera assembly in the self - moving device can be determined according to the working scenario, so as to meet the special requirements of the self - moving device for functions such as positioning (e.g., visual SLAM) and obstacle avoidance (visual obstacle detection) in different working scenarios.

[0087] In some embodiments, optionally, step E2 may specifically include the following steps F1 - F2:

[0088] F1, if the working scenario is daytime, determine that the working mode of the camera assembly is a low - illumination camera and a binocular camera in the first state, or only a binocular camera in the first state; wherein, the first state is that the infrared laser is in the off state.

[0089] Specifically, if the working scenario is daytime, passive binocular vision can be achieved by turning off the infrared laser. At this time, the binocular camera can be used to calculate the visual SLAM and depth map of the charging pile. Thus, the requirements of the self - moving device for positioning and obstacle avoidance functions during the day can be met only by using the binocular camera. Among them, the depth map is mainly used for visual obstacle detection and can also assist in visual SLAM. Further, on the basis of passive binocular vision, a low - illumination camera can also be used in combination to help improve the positioning accuracy of the self - moving device by combining different dimensions of positioning information.

[0090] F2, if the working scenario is night, determine that the working mode of the camera assembly is a low - illumination camera and a binocular camera in the second state; wherein, the second state is that the infrared laser is in the on state.

[0091] Specifically, if the working scenario is night, active binocular vision can be achieved by turning on the infrared laser. At this time, the image of the binocular camera is a speckle image and cannot be used to calculate the visual SLAM of the charging pile, and can only be used to calculate the depth map. Therefore, the requirement of the self - moving device for the positioning function at night cannot be met. If you want to simultaneously meet the requirements of the self - moving device for positioning and obstacle avoidance functions at night, a low - illumination camera needs to be used in combination on the basis of active binocular vision, and the visual SLAM information of the charging pile is provided by the low - illumination camera.

[0092] See Figure 8 , the controller in the self - moving device may specifically include a two - dimensional feature point extraction module and a camera assembly parameter calibration module. Among them, the two - dimensional feature point extraction module is used to obtain the target image corresponding to the charging pile collected by the camera assembly, and extract feature points from the target image based on preset feature information to obtain two - dimensional feature points. The camera assembly parameter calibration module is used to calibrate the parameters of the camera assembly based on the correspondence between the two - dimensional feature points and the three - dimensional feature points.

[0093] Among them, the three-dimensional feature points are the feature points pre-calibrated on the charging pile pre-stored by the controller based on preset feature information. Optionally, the preset feature information includes structural information and additional flag information. Among them, the structural information includes at least one of the following: edge information, corner point information, and texture information. The additional flag information includes preset pattern information, where the preset pattern information includes two-dimensional code information, bar code information, etc.

[0094] In some embodiments, optionally, the two-dimensional feature point extraction module is configured to: extract feature points from the first image based on preset feature information to obtain the first image two-dimensional feature points corresponding to the first image; and extract feature points from the second image based on preset feature information to obtain the second image two-dimensional feature points corresponding to the second image. Among them, the first image is the charging pile image collected by the left camera in the binocular camera (camera assembly), and the second image is the charging pile image collected by the right camera in the binocular camera (camera assembly).

[0095] In some embodiments, optionally, the camera assembly parameter calibration module is configured to: determine the first internal parameters of the left camera based on the correspondence between the first image two-dimensional feature points and the three-dimensional feature points; determine the second internal parameters of the right camera based on the correspondence between the second image two-dimensional feature points and the three-dimensional feature points. Among them, the internal parameters include focal length and optical center.

[0096] In some embodiments, optionally, the camera assembly parameter calibration module is further configured to: determine the first external parameters of the left camera based on the correspondence between the first image two-dimensional feature points and the three-dimensional feature points and the first internal parameters; where the first external parameters are used to represent the relative pose information between the left camera and the charging pile; determine the second external parameters of the right camera based on the correspondence between the second image two-dimensional feature points and the three-dimensional feature points and the second internal parameters; where the second external parameters are used to represent the relative pose information between the right camera and the charging pile; determine the third external parameters of the binocular camera according to the first external parameters and the second external parameters; where the third external parameters are used to represent the relative pose information between the left camera and the right camera.

[0097] See Figure 9, the controller in the self - moving device may further include a working - scenario determination module for the self - moving device and a working - mode determination module for the camera assembly. Among them, the working - scenario determination module for the self - moving device is configured to: determine the working scenario of the self - moving device based on the working time of the self - moving device or the ambient brightness where the self - moving device is located. The working - mode determination module for the camera assembly is configured to: if the working scenario is daytime, determine the working mode of the camera assembly as a low - illumination camera and a binocular camera in a first state, or only a binocular camera in the first state; where the first state is that the infrared laser is in the off state; if the working scenario is night, determine the working mode of the camera assembly as a low - illumination camera and a binocular camera in a second state; where the second state is that the infrared laser is in the on state.

[0098] The foregoing has shown and described the basic principles, main features, and advantages of the present application. Those skilled in the art should understand that the above - mentioned embodiments do not limit the present application in any form. Any technical solutions obtained by means of equivalent replacement or equivalent transformation fall within the protection scope of the present application.

Claims

1. A self - moving device system for camera component parameter configuration, including a self - moving device and a charging pile, where the charging pile is used to charge the self - moving device; the self - moving device includes: Body; Traveling wheel assembly for supporting the body; Camera assembly mounted to the body for collecting images around the self - moving device; Controller communicatively connected to the camera assembly, characterized in that the controller prestores three - dimensional feature points on the charging pile; wherein, the three - dimensional feature points are pre - calibrated based on preset feature information; The controller is configured to: Obtain a target image corresponding to the charging pile collected by the camera assembly, and extract feature points from the target image based on the preset feature information to obtain two - dimensional feature points; Calibrate the parameters of the camera assembly based on the correspondence between the two - dimensional feature points and the three - dimensional feature points.

2. The self - moving device system according to claim 1, characterized in that, The camera assembly includes a binocular camera, and the binocular camera includes a left camera and a right camera.

3. The self - moving device system according to claim 2, characterized in that, The target image includes a first image and a second image, the first image is the image collected by the left camera, and the second image is the image collected by the right camera.

4. The self - moving device system according to claim 3, wherein, Extracting feature points from the target image based on the preset feature information to obtain two - dimensional feature points includes: Extracting feature points from the first image based on the preset feature information to obtain first - image two - dimensional feature points corresponding to the first image; and, Extracting feature points from the second image based on the preset feature information to obtain second - image two - dimensional feature points corresponding to the second image.

5. The self - moving device system according to claim 4, wherein, Calibrating the parameters of the camera assembly based on the correspondence between the two - dimensional feature points and the three - dimensional feature points includes: Determining a first internal parameter of the left camera based on the correspondence between the first - image two - dimensional feature points and the three - dimensional feature points; Determining a second internal parameter of the right camera based on the correspondence between the second - image two - dimensional feature points and the three - dimensional feature points.

6. The self - moving device system according to claim 5, wherein, The internal parameter includes focal length and optical center.

7. The self-moving device system according to claim 5, characterized in that Calibrating the parameters of the camera assembly based on the correspondence between the two - dimensional feature points and the three - dimensional feature points includes: Determining a first external parameter of the left camera based on the correspondence between the first - image two - dimensional feature points and the three - dimensional feature points and the first internal parameter; wherein, the first external parameter is used to characterize the relative pose information between the left camera and the charging pile; Determining a second external parameter of the right camera based on the correspondence between the second - image two - dimensional feature points and the three - dimensional feature points and the second internal parameter; wherein, the second external parameter is used to characterize the relative pose information between the right camera and the charging pile; Determining a third external parameter of the binocular camera according to the first external parameter and the second external parameter; wherein, the third external parameter is used to characterize the relative pose information between the left camera and the right camera.

8. The self - moving device system according to any one of claims 1 - 7, characterized in that, The preset feature information includes structural information and additional flag information.

9. The self - moving device system according to claim 8, characterized in that, The structural information includes at least one of the following: edge information, corner point information, and texture information.

10. The self - moving device system according to claim 8, characterized in that, The additional flag information includes preset pattern information, and the preset pattern information includes two - dimensional code information.

11. The self - moving device system according to claim 2, characterized in that, The self - moving device system further includes an infrared laser, and the camera assembly further includes a low - illumination camera.

12. The self - moving device system according to claim 11, characterized in that, The controller is further used for: Determine the working scenario of the self - moving device, where the working scenario is day or night; Determine the working mode of the camera component in the self - moving device according to the working scenario.

13. The self-moving device system according to claim 12, characterized in that, Determining the working scenario of the self - moving device includes: Based on the working time of the self - moving device or the ambient brightness where the self - moving device is located, determine the working scenario of the self - moving device.

14. The self - moving device system according to claim 12 or 13, characterized in that, Determining the working mode of the camera component in the self - moving device according to the working scenario includes: If the working scenario is day, determine that the working mode of the camera component is a low - light camera and a binocular camera in the first state, or only the binocular camera in the first state; where the first state is that the infrared laser is in the off state; If the working scenario is night, determine that the working mode of the camera component is a low - light camera and a binocular camera in the second state; where the second state is that the infrared laser is in the on state.

Images