Binocular camera, vision system, intelligent wheelchair, and data processing method

Abstract

Provided in the present application are a binocular camera, a vision system, an intelligent wheelchair, and a data processing method. The binocular camera comprises: a binocular lens, which is used for collecting disparity image data regarding a target scene, and outputting the disparity image data; and a data processing chip, which is electrically connected to the binocular lens and is used for processing the disparity image data to obtain a point cloud, converting the point cloud into two-dimensional data, and outputting same, wherein the two-dimensional data is used for representing the position of an obstacle in the target scene. Compared with a three-dimensional point cloud, the amount of data is reduced, such that the time for a binocular camera outputting data can be reduced, and a storage space of the binocular camera and a storage space of a carrier device which carries the binocular camera can also be reduced. Therefore, the present application is applicable to a wide range and requires relatively low costs.

Description

Binocular camera, vision system, intelligent wheelchair and data processing method

[0001] The present application claims priority to the Chinese patent application No. 2024105466707, filed on April 30, 2024, and entitled "Binocular camera, vision system, intelligent wheelchair and data processing method", the content of which is incorporated herein by reference in its entirety. TECHNICAL FIELD

[0002] The present application relates to a binocular camera, a vision system, an intelligent wheelchair and a data processing method, but is not limited thereto. BACKGROUND

[0003] In recent years, with the continuous progress of hardware technology and the continuous optimization of algorithms, binocular cameras that can directly output point cloud maps have appeared on the market. Such binocular cameras can calculate and output point cloud maps in real time during the shooting process through built-in high-performance processors and advanced image processing algorithms, greatly improving the efficiency and accuracy of three-dimensional reconstruction. However, since the point cloud map is composed of a large number of three-dimensional data points, its data volume is usually very large.

[0004] This not only puts higher requirements on the storage and transmission capacity of the camera, but also brings challenges to subsequent point cloud processing and analysis. SUMMARY

[0005] Some embodiments of the present application provide a binocular camera, comprising:

[0006] A binocular camera is configured to capture a target scene and output parallax image data.

[0007] A data processing chip is electrically connected to the binocular camera and configured to process the parallax image data to obtain a point cloud, and convert the point cloud into two-dimensional data for output, the two-dimensional data being used to represent the position of an obstacle in the target scene.

[0008] In some embodiments, the two-dimensional data includes a plurality of data points in polar coordinates; the plurality of data points in polar coordinates are used to represent the position of the obstacle.

[0009] In some embodiments, the two-dimensional data further includes a radar chart; the radar chart includes a plurality of rays at different angles, the length of the ray being used to represent whether there is an obstacle, and the length of the ray and the angle of the ray being used to represent the position of the obstacle when there is an obstacle.

[0010] In some embodiments, the binocular camera further receives external parameters, the external parameters being used to set an obstacle filtering condition, and the two-dimensional data output by the binocular camera does not include an obstacle satisfying the obstacle filtering condition.

[0011] In some embodiments, the external parameters include a size of a mounting device on which the binocular camera is mounted or a size of an obstacle that can be passed.

[0012] In some embodiments, the binocular camera further comprises a communication module.

[0013] The communication module is electrically connected with the data processing chip, and the communication module is configured to be connected with an external microprocessor.

[0014] In some embodiments, the binocular camera further comprises a fill light; the fill light provides fill light.

[0015] In some embodiments, the binocular camera further comprises a printed circuit board.

[0016] The binocular camera, the data processing chip and the communication module are located on the printed circuit board.

[0017] In some embodiments, the data processing chip is a chip without AI function.

[0018] Some embodiments of the present application provide a visual system, the visual system comprising a single-chip microcomputer and the binocular camera involved in the above-mentioned embodiments.

[0019] In some embodiments, the microprocessor comprises a single-chip microcomputer.

[0020] Some embodiments of the present application provide an intelligent wheelchair, comprising a seat, a backrest, an armrest and the visual system involved in the above-mentioned embodiments, and the armrest end is provided with the binocular camera.

[0021] In some embodiments, the armrest end comprises a housing and the binocular camera, the binocular camera is located in the housing, and the armrest end further comprises a display screen, the display screen is located in the housing.

[0022] In some embodiments, the housing comprises an upper housing structure and a lower housing structure.

[0023] The upper housing structure is provided with a first cavity, and the lower housing structure is provided with a second cavity, the first cavity is configured to mount the display screen, and the second cavity is configured to mount the binocular camera.

[0024] When the upper housing structure and the lower housing structure cooperate, the first cavity is located above the second cavity.

[0025] Some embodiments of the present application provide a processing method of a binocular camera, the binocular camera comprising a binocular camera, the method comprising:

[0026] Obtaining disparity image data output by the binocular camera;

[0027] After processing the disparity image data, a point cloud is obtained, and the point cloud is converted into two-dimensional data, the two-dimensional data being configured to represent a position of an obstacle in a target scene.

[0028] In some embodiments, the parallax image data is processed to obtain a point cloud, and the point cloud is converted into two-dimensional data, specifically including:

[0029] The parallax image data is processed using a preset algorithm to obtain a point cloud, and the point cloud is processed to obtain obstacle data;

[0030] The plane data of the obstacle is obtained by projecting the obstacle data, and the two-dimensional data is generated according to the plane data of the obstacle.

[0031] In some embodiments, after the point cloud is processed to obtain the obstacle data, the method further includes:

[0032] The obstacle data is filtered using a preset obstacle filtering condition;

[0033] Correspondingly, the plane data of the obstacle is obtained by projecting the obstacle data, specifically including:

[0034] The plane data of the obstacle is obtained by projecting the filtered obstacle data.

[0035] In some embodiments, one obstacle type corresponds to one obstacle filtering condition, and each obstacle data includes obstacle three-dimensional data and obstacle type;

[0036] Correspondingly, the obstacle data is filtered using a preset obstacle filtering condition, specifically including:

[0037] For each obstacle, if the three-dimensional data of the obstacle meets the filtering condition corresponding to the obstacle type, the three-dimensional data of the obstacle is deleted to obtain the filtered obstacle data.

[0038] In some embodiments, if the obstacle type is a pit type, the filtering condition of the obstacle includes that the width of the obstacle along the forward direction of the carrying device is less than a preset width threshold;

[0039] The preset width threshold is determined according to the wheel size of the carrying device.

[0040] In some embodiments, if the obstacle type is a protrusion type, the filtering condition of the obstacle includes that the height of the obstacle is less than a preset height threshold, and the preset height threshold is determined according to the chassis height of the carrying device.

[0041] In some embodiments, if the obstacle type is a crossing type, the filtering condition of the obstacle includes that the upper boundary height of the crossable area of the obstacle is greater than the height of the carrying device, the lower boundary height of the crossable area of the obstacle is less than the chassis height of the carrying device, and the width of the crossable area of the obstacle is greater than the width of the carrying device.

[0042] In some embodiments, the two-dimensional data is generated according to the planar data of the obstacle, specifically including:

[0043] The profile data of the obstacle is generated according to the planar data of the obstacle.

[0044] The intersection points of the rays of different angles and the profile data of the obstacle are calculated to obtain a plurality of data points in a polar coordinate system.

[0045] The binocular camera, the visual system, the intelligent wheelchair and the data processing method provided by the application, the binocular camera includes a binocular camera and a data processing chip. The binocular camera is used for collecting and outputting parallax image data of a target scene, and the data processing chip is used for converting the parallax image data into point cloud and converting the point cloud into two-dimensional data output. The two-dimensional data is used to represent the position of the obstacle in the target scene. Compared with the data amount of three-dimensional point cloud, the time of the binocular camera outputting data externally can be shortened, the storage space of the binocular camera and the carrying device carrying the binocular camera can be reduced, the application range is wide, and the cost is lower. BRIEF DESCRIPTION OF DRAWINGS

[0046] The drawings incorporated into the specification and constituting a part of the specification, show embodiments consistent with the application, and together with the specification, serve to explain the principles of the application.

[0047] Fig. 1 is a schematic diagram of the architecture of the binocular camera provided by some embodiments of the application;

[0048] Fig. 2 is a schematic diagram of two-dimensional data and a radar chart provided by some embodiments of the application;

[0049] Fig. 3 is a schematic diagram of a radar chart of a display area provided by some embodiments of the application;

[0050] Fig. 4 is a schematic diagram of a data processing method of the binocular camera provided by some embodiments of the application;

[0051] Fig. 5 is a schematic diagram of a pit type obstacle provided by some embodiments of the application;

[0052] Fig. 6 is a schematic diagram of a convex type obstacle provided by some embodiments of the application;

[0053] Fig. 7 is a schematic diagram of a through type obstacle provided by some embodiments of the application;

[0054] Fig. 8 is a schematic diagram of the structure of the intelligent wheelchair provided by some embodiments of the application;

[0055] Fig. 9 is a schematic diagram of the intelligent wheelchair provided by an embodiment of the application;

[0056] Fig. 10 is a schematic diagram of the installation of the binocular camera and the display screen provided by some embodiments of the application.

[0057] Reference signs: 300, binocular camera; 310, binocular camera; 320, data processing chip; 320, communication module; 410, upper shell structure; 420, lower shell structure; 421, first cavity; 422, second cavity; 432, protective back cover; 500, audio and video camera; X, transverse movement direction; Y, forward direction.

[0058] The specific embodiments of the present application have been shown by the above drawings, and will be described in more detail hereinafter. These drawings and the written description are not intended to restrict the scope of the present application in any way, but to illustrate the concept of the present application by referring to specific embodiments. DETAILED DESCRIPTION

[0059] The exemplary embodiments will be described in detail herein with reference to the attached drawings. In the following description, the same numbers are used to indicate the same or similar components. The embodiments described in the following exemplary embodiments are not meant to represent all embodiments consistent with the present application. Rather, they are merely examples of apparatus and methods consistent with some aspects of the present application.

[0060] In the description of the present application, the terms "first", "second" are used only for the purpose of description, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first", "second" can explicitly or implicitly include one or more of the features. In the description of the present application, the meaning of "a plurality of" is two or more, unless otherwise specifically limited.

[0061] In the description of the present application, it should be noted that, unless otherwise specifically defined and limited, the terms "connected", "connected" should be understood broadly, for example, it can be fixedly connected, or detachably connected, or integrally connected; it can be mechanically connected, or electrically connected; it can be directly connected, or indirectly connected through an intermediate medium. For those skilled in the art, the specific meaning of the above terms in the present application can be understood according to the specific circumstances.

[0062] It should be noted that the user information (including but not limited to user equipment information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data authorized by the user or authorized by all parties, and the collection, use and processing of related data need to comply with relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation portal for user to choose authorization or refusal.

[0063] With the rapid development of machine vision and computer technology, binocular camera as a camera system simulating human binocular vision has been widely used in robot navigation, three-dimensional measurement, virtual reality and other fields. Binocular camera shoots the target object through two cameras at the same time, calculates the distance between the object and the camera by using the principle of parallax, and then realizes three-dimensional reconstruction.

[0064] In the application of binocular camera, point cloud map as an important three-dimensional data representation form can intuitively show the spatial structure and shape of the target object. The traditional binocular camera system usually needs to go through a series of complex image processing steps such as image acquisition, camera calibration, feature extraction, image matching, etc. to finally generate point cloud map. However, a large amount of data calculation and processing is involved in this process, which is time-consuming and laborious.

[0065] In recent years, with the continuous progress of hardware technology and the continuous optimization of algorithm, there have been binocular cameras on the market that can directly output point cloud map. This kind of binocular camera can calculate and output point cloud map in real time during shooting through the built-in high-performance processor and advanced image processing algorithm, greatly improving the efficiency and accuracy of three-dimensional reconstruction. However, since point cloud map is composed of a large number of three-dimensional data points, its data volume is usually very large.

[0066] This not only puts higher requirements on the storage and transmission capacity of the camera, but also brings challenges to the subsequent point cloud processing and analysis. More specifically, when using existing binocular cameras on a certain backend application device, a processing chip needs to be set up in the backend application device, which runs processing algorithms. More specifically, three-dimensional point cloud data is converted into obstacle data, and the distance between the camera and the obstacle is extracted, which can be used for application. Since the data output by the existing binocular camera cannot directly represent the object, the backend application device also needs to do algorithm processing. The processing chip in the backend application device needs to be a high-power chip, which requires high hardware resources, and ordinary single-chip microcomputers cannot be used as processing chips. In addition, the existing binocular camera has algorithms and logic controls, and the backend application device also has algorithms and logic controls, which have redundant resources. In addition, some existing binocular cameras are equipped with AI-enabled processing chips, resulting in high cost of existing binocular cameras.

[0067] Some embodiments of the present application provide a new binocular camera, vision system, intelligent wheelchair and data processing method, which can greatly reduce the amount of data output by the binocular camera by converting the point cloud obtained by the binocular camera into two-dimensional data output. Since the data volume is small, SPI / URAT / I2C protocol can be used for data transmission. Since a fill light is used, it can be applied to indoor and outdoor scenes.

[0068] In the binocular camera application, the two-dimensional data can directly represent the size of the obstacle and the position information of the obstacle. The rear-end application device does not need complex algorithms, and the processing chip of the rear-end application device can be replaced by a microprocessor. For example, the rear-end application device can only need to configure a single-chip microcomputer to process the two-dimensional data output by the binocular camera. Since the binocular camera can output two-dimensional data with a small amount of data, the requirement for the rear-end application device is not high when applied to the rear-end application device, and the applicability is wider. The laser radar with wider applicability but higher cost can be replaced to reduce the hardware cost of the rear-end application device. Since the binocular camera does not need to identify the specific type of the obstacle, for example, it does not need to identify whether the obstacle is a table or a chair, which makes the processing chip without AI function in the binocular camera can further reduce the cost.

[0069] In addition, in the binocular camera application, the size of the obstacle that can be passed through the device carrying the binocular camera can be directly input to complete the configuration of the obstacle filtering condition, which can be applied to various scenes of using the binocular camera.

[0070] FIG. 1 is a structural schematic diagram of a binocular camera 300 provided by some embodiments of the present application. As shown in FIG. 1, the binocular camera 300 provided by some embodiments of the present application includes a binocular camera 310 and a data processing chip 320.

[0071] The binocular camera 310 and the data processing chip 320 are electrically connected. The binocular camera 310 collects and outputs parallax image data of a target scene, and the data processing chip 320 is used to convert the parallax image data into point cloud and convert the point cloud into two-dimensional data output. The two-dimensional data is used to represent the position of the obstacle in the target scene.

[0072] In the above technical solution, the binocular camera 300 includes the binocular camera 310 and the data processing chip 320. The binocular camera 310 is used to collect and output parallax image data of a target scene, and the data processing chip 320 is used to convert the parallax image data into point cloud and convert the point cloud into two-dimensional data output. The two-dimensional data is used to represent the obstacle in the target scene. Compared with the three-dimensional point cloud, the data amount is reduced, the time for the binocular camera 300 to output data externally can be shortened, the storage space of the binocular camera and the device carrying the binocular camera can be reduced, the application range is wide, and the cost is lower.

[0073] In some embodiments, the binocular camera is a device with two cameras, which is used to simulate the stereoscopic vision effect of human eyes. The parallax image data is obtained by simultaneously acquiring the images of the same object by the two cameras. The data processing chip 320 can match the parallax image data to obtain the distance information of the object to the binocular camera, and then output the three-dimensional point cloud.

[0074] In some embodiments, the two-dimensional data includes a plurality of data points in polar coordinates, and the plurality of data points are used to represent the positions of the obstacles in the target scene. Each data point includes an angle value and a length value. In some embodiments, if the length value is a preset value, it means that there is no obstacle at the angle. If the length value is not the preset value, it means that there is an obstacle at the angle, and the position of the obstacle is the position of the length value. In some other embodiments, if the length value is greater than a preset value, it means that there is no obstacle at the angle. If the length value is less than the preset value, it means that there is an obstacle at the angle, and the position of the obstacle is the position of the length value.

[0075] In the following example, the two-dimensional data includes 180 data points in polar coordinates. The data point Pi is (ai, Li), 1≤i≤180, and the angle value ai is the angle between the line connecting the data point and the origin and the a-axis, which is a coordinate axis extending horizontally to the right. FIG. 2 is a schematic diagram of part of the two-dimensional data provided by some embodiments of the present application, and only 11 data points are shown in FIG. 2. The 11 data points are labeled as P1 to P11. The length values of the data points P1, P2, P4, and P6 are all preset values, and there is no obstacle in the direction of the angles a1, a2, a4, and a6. The length values of the data points P3, P7, and P8 are all less than the preset value, that is, there is an obstacle in the direction of the angles a3, a7, and a8. The specific position of the obstacle can be determined according to the lengths of the third and seventh rays. That is, the obstacle is located in the region of the two points (a3, L3) and (a7, L7), and (a3, L3) and (a7, L7) are the contour points of the obstacle.

[0076] In some embodiments, the two-dimensional data further includes a radar chart, and the radar chart includes a plurality of rays at different angles. The length of the ray is used to represent whether there is an obstacle, and the length of the ray and the angle of the ray together represent the position of the obstacle when there is an obstacle.

[0077] In some embodiments, the data processing chip 320 further generates a radar chart based on the plurality of data points in polar coordinates. More specifically, as shown in FIG. 2, the origin is connected to each data point to form a plurality of rays at different angles, and thus the radar chart is obtained.

[0078] In some embodiments, for the radar chart shown in FIG. 2, a display region is preset, and the rays in the display region are displayed, as shown in FIG. 3. Thus, the covered position of the rays in the display region is the drivable region, and the position not covered by the rays in the display region is the obstacle region.

[0079] In some embodiments, the binocular camera comprises a fill light, and the fill light is configured to provide fill light. The fill light provides a planar array of point light sources, and the binocular camera 310 generates three-dimensional data by identifying distortions in the planar array of point light sources when projected onto the target scene. The fill light can be turned on when the binocular camera 310 cannot detect obstacles, and more specifically, the fill light provides fill light for the binocular camera 310 when the ambient brightness of the target scene is less than a predetermined threshold. In this way, the binocular camera 300 improves the detection accuracy in dark environments and is applicable to a wider range of scenes.

[0080] In some embodiments, the binocular camera further comprises a CMOS light sensing device, and the CMOS light sensing device is configured to implement photoelectric conversion and convert optical signals into image data.

[0081] In some embodiments, the binocular camera 300 further comprises a communication module 320, and the communication module 320 is electrically connected to the data processing chip 320. The communication module 320 is also configured to connect to an external microprocessor, and the communication module 320 is configured to output two-dimensional data externally. Since the amount of two-dimensional data is relatively small, the communication module 320 can use I2C protocol or SPI protocol for data transmission, but is not limited to the above protocols, and can also use USB protocol for data transmission. The device does not need complex algorithms, and the processing chip of the device can be replaced with a microprocessor with small computing power. More specifically, the external microprocessor can be a single-chip microcomputer.

[0082] In some embodiments, the binocular camera 300 further comprises a printed circuit board (PCB), and the binocular camera 310, the communication module 320, and the data processing chip 320 are located on the printed circuit board. The printed circuit board is configured to implement the connection between the binocular camera 310 and the data processing chip 320.

[0083] In some embodiments, the data processing chip 320 obtains point cloud data after processing the parallax image data, uses a preset algorithm to process the parallax image data to obtain point cloud data, processes the point cloud data to obtain obstacle data, projects the obstacle data to obtain plane data of the obstacle, and generates two-dimensional data based on the plane data of the obstacle.

[0084] In some embodiments, in order to reduce the amount of data calculation, the binocular camera 300 further receives external parameters, and the external parameters are configured to set obstacle filtering conditions. The two-dimensional data output by the binocular camera 300 does not include the position of the obstacle that meets the obstacle filtering conditions.

[0085] More specifically, the data processing chip 320 aggregates and identifies the point cloud to obtain obstacle data, filters the obstacle data using a preset obstacle filtering condition, projects the filtered obstacle data to obtain plane data of the obstacle, and generates two-dimensional data based on the plane data of the obstacle. When the two-dimensional data is generated based on the filtered obstacle data, the two-dimensional data does not indicate the presence of the filtered obstacle in the target scene.

[0086] That is, if the obstacle is too small to hinder the continuation of the travel of the device, the binocular camera directly filters out the obstacle, and the output two-dimensional data does not contain the obstacle data. When the binocular camera is mounted on the device, the device does not need to plan a path to avoid the obstacle, and the calculation amount of the device can be reduced.

[0087] In some embodiments, the data processing chip is a chip without AI function. The AI function refers to the ability to identify the specific type of the obstacle, for example, without identifying the specific type of the obstacle as a door, a staircase, a road curb, a table, a chair, etc.

[0088] In some embodiments, the external parameters include the size of the device on which the binocular camera 300 is mounted or the size of the obstacle that can be passed by the device, which can adapt to the use scenario of the mounted device. The size of the device is used to calculate the size of the obstacle that can be passed by the device.

[0089] In some embodiments, the device can be a vehicle, a robot, or a smart wheelchair, which is only an example and does not limit the device to the above devices.

[0090] In summary, the binocular camera 300 provided by the present application has the following advantages:

[0091] 1. Reducing the calculation complexity, since the two-dimensional data is compared with the three-dimensional point cloud, the back-end application device on which the binocular camera is mounted can process the two-dimensional data faster, which is particularly important for the back-end application device that needs real-time processing, such as automatic driving and navigation of a vehicle.

[0092] 2. Simplifying data processing and analysis, in the back-end application device, the existing multi-image processing and machine learning algorithms can be directly used to process the two-dimensional data, without the need to develop algorithms for three-dimensional point clouds, thereby simplifying data processing and analysis.

[0093] 3. Reducing storage space, the two-dimensional data occupies less storage space than the three-dimensional point cloud data, which is convenient for storage and transmission. The cost of the binocular camera and the back-end application device on which the binocular camera is mounted can be greatly reduced.

[0094] 4、Directly represent environmental information, two-dimensional data can directly represent obstacles, and the back-end application device does not need complex algorithms. The processing chip of the back-end application device can be replaced with a microprocessor, for example, a single-chip microcomputer can be used.

[0095] 5、Compared with the binocular camera 300 with AI function, it is not necessary to identify what kind of object the obstacle is, but only to identify which category the obstacle is, for example, it is not necessary to identify whether the obstacle is a door, a staircase, a road curb, a table, a chair, etc., but only to identify the obstacle type, for example, a concave type, a convex type or a crossing type, and the cost is lower.

[0096] 6、Since the binocular camera 300 can output two-dimensional data with a small amount of data, the requirement for the back-end application device is not high when applied to the back-end application device, the applicability is wider, the laser radar with wider applicability but higher cost can be replaced, and the hardware cost of the back-end application device is reduced.

[0097] As shown in FIG. 4, some embodiments of the present application provide a data processing method of a binocular camera 300, the method comprising the following steps:

[0098] S101, obtaining parallax image data output by an image sensor.

[0099] The binocular camera 310 is used to collect a target scene and output parallax image data. The binocular camera is a device with two cameras, which is used to simulate the stereoscopic visual effect of the human eye. It obtains parallax image data by recording the images of the same object by two cameras at the same time.

[0100] S102, obtaining point cloud after processing the parallax image data, and converting the point cloud into two-dimensional data.

[0101] More specifically, the parallax image data is matched to obtain distance information of the object to the binocular camera, and then three-dimensional point cloud is output. The two-dimensional data is used to represent the position of the obstacle.

[0102] In the above technical solution, the binocular camera 300 comprises a binocular camera 310 and a data processing chip 320. The binocular camera 310 collects a target scene and obtains parallax image data of the target scene, and the data processing chip 320 is used to convert the parallax image data into point cloud, and convert the point cloud into two-dimensional data output. Compared with the data amount of three-dimensional point cloud, the time for the binocular camera 300 to output data externally can be shortened, and the storage space of the binocular camera and the device carrying the binocular camera can be reduced.

[0103] Some embodiments of the present application provide a data processing method of a binocular camera 300, which comprises:

[0104] S201, obtain parallax image data output by an image sensor.

[0105] S202, obtain point cloud data by processing the parallax image data using a preset algorithm, and process the point cloud data to obtain obstacle data.

[0106] The point cloud data can be obtained by processing the parallax image data using an existing algorithm, which will not be described herein. The obstacle data can be obtained by aggregating and identifying the point cloud data using an existing aggregation algorithm and an identification algorithm.

[0107] S203, project the obstacle data to obtain plane data of the obstacle, and generate two-dimensional data according to the plane data of the obstacle.

[0108] The obstacle data is projected on a horizontal plane to obtain the plane data of the obstacle, and the plane data of the obstacle includes:

[0109] The obstacle data on the horizontal plane at different heights is projected onto a projection plane, which is also a plane parallel to the ground. The horizontal plane is a plane parallel to the ground. The projection data is de-duplicated to obtain the plane data of the obstacle.

[0110] If the two-dimensional data includes a plurality of data points in polar coordinates, the two-dimensional data is generated according to the plane data of the obstacle, and the two-dimensional data includes:

[0111] The contour point data of the obstacle is selected from the plane data of the obstacle, the contour line of the obstacle is fitted according to the contour point data of the obstacle, the intersection of the rays at different angles and the contour line of the obstacle is calculated, and the intersection is converted into a plurality of data points in a polar coordinate system.

[0112] In some embodiments, if the two-dimensional data is a plurality of data points in a rectangular coordinate system, the contour point data of the obstacle can be selected from the plane data of the obstacle as the two-dimensional data.

[0113] Some embodiments of the present application provide a data processing method of a binocular camera 300, which includes:

[0114] S301, obtain parallax image data output by an image sensor.

[0115] S302, obtain point cloud data by processing the parallax image data using a preset algorithm, and aggregate and identify the point cloud data to obtain obstacle data.

[0116] The point cloud data can be obtained by processing the parallax image data using an existing algorithm, which will not be described herein. The obstacle data can be obtained by aggregating and identifying the point cloud data using an existing aggregation algorithm and an identification algorithm.

[0117] S303, filtering the obstacle data using a preset obstacle filtering condition.

[0118] If the obstacle does not affect the passage of the carrying device, the obstacle data can be filtered out. The obstacle filtering condition is set according to the passage ability of the carrying device.

[0119] S304, projecting the filtered obstacle data to obtain plane data of the obstacle, and generating two-dimensional data according to the plane data of the obstacle.

[0120] In the above technical solution, the obstacle data is filtered by the obstacle filtering condition, so that the data processing amount of the binocular camera 300 and the data processing amount of the carrying device can be reduced, and a more accurate drivable area can be obtained.

[0121] Some embodiments of the present application also provide a data processing method of a binocular camera 300, which comprises the following steps, specifically comprising:

[0122] S401, obtaining disparity image data output by an image sensor.

[0123] S402, obtaining point cloud by processing the disparity image data using a preset algorithm.

[0124] The point cloud can be obtained by processing the disparity image data using an existing algorithm, which will not be described here.

[0125] S403, obtaining obstacle data by aggregating and identifying the point cloud.

[0126] The obstacle data can be obtained by aggregating and identifying the point cloud using an existing aggregation algorithm and an identification algorithm, and the obstacle data includes three-dimensional data of at least one obstacle and a type of the obstacle.

[0127] The three-dimensional data of the obstacle includes three-dimensional coordinate points, which can be three-dimensional coordinate points of the center of the obstacle, three-dimensional coordinate points on the contour of the obstacle, or some three-dimensional coordinate points representing features of the obstacle. The type of the obstacle includes a pit type, a protrusion type, and a through type.

[0128] S404, for each obstacle, if the three-dimensional data of the obstacle meets a filtering condition corresponding to the type of the obstacle, deleting the three-dimensional data of the obstacle to obtain filtered obstacle data.

[0129] The types of the obstacles are different, and the filtering conditions of the obstacles are different accordingly. The filtering conditions of the obstacles are used to filter out some obstacle data.

[0130] For example, if the obstacle type is a pit type, the width of the obstacle can be directly filtered out if the width is relatively small. If the obstacle type is a bump type, the height of the obstacle can be directly filtered out if the height is relatively small. If the obstacle type is a crossing type, the upper boundary of the crossable region of the obstacle is relatively high, the lower boundary is relatively low, and the width of the crossable region of the obstacle is relatively large, and the obstacle can be directly filtered out.

[0131] After determining the obstacle filtering condition, the three-dimensional data of the obstacle can be filtered using the obstacle filtering condition. If the three-dimensional data of the obstacle meets the obstacle filtering condition, the three-dimensional data of the obstacle is deleted, and filtered obstacle data is obtained, and a more accurate drivable region can be obtained.

[0132] In S405, the three-dimensional data of each obstacle in the filtered obstacle data is projected onto a horizontal plane to obtain plane data of the obstacle, and two-dimensional data is generated according to the plane data of the filtered obstacle.

[0133] The three-dimensional data of the obstacle includes a three-dimensional coordinate point (x1, y1, z1), and the plane data of the obstacle includes a two-dimensional coordinate (x2, y2). For the three-dimensional data of each obstacle in the filtered obstacle data, the X coordinate and the Y coordinate in the three-dimensional data of the obstacle are taken as the X coordinate and the Y coordinate in the plane data of the obstacle.

[0134] In some embodiments, if the obstacle type is a pit type, the filtering condition of the obstacle includes that the width W of the obstacle along the forward direction Y of the carrying device is less than a preset width threshold.

[0135] The preset width threshold is determined according to the wheel size of the carrying device input externally.

[0136] For a pit type obstacle, as shown in FIG. 5, the width W of the obstacle along the forward direction Y of the carrying device affects the passing of the carrying device through the obstacle. If the width W of the obstacle along the forward direction Y of the carrying device is relatively large, the carrying device will fall into the pit. The width of the pit that the carrying device can cross is determined based on the wheel size of the carrying device, that is, the preset width threshold is determined. When the width W of the obstacle along the forward direction Y of the carrying device is less than the preset width threshold, it indicates that the carrying device can pass through the obstacle, and the obstacle can be filtered out. When the two-dimensional data is generated based on the filtered obstacle data, the two-dimensional data does not indicate that the filtered obstacle exists in the target scene.

[0137] In some embodiments, if the obstacle type is a bump type, the filtering condition of the obstacle includes that the height H of the obstacle is less than a preset height threshold, and the preset height threshold is determined according to the ride height of the carrying device.

[0138] For the convex type of obstacles, as shown in FIG. 6, the height of the obstacle will affect the passing of the carrying device. If the obstacle is relatively high and the chassis height of the carrying device is relatively low, the carrying device cannot pass the obstacle. Therefore, based on determining the preset height threshold according to the chassis height of the carrying device, the height H of the obstacle is less than the preset height threshold, and the carrying device can pass the obstacle, the obstacle can be filtered out. When generating two-dimensional data based on the filtered obstacle data, the two-dimensional data will not indicate that the filtered obstacle exists in the target scene.

[0139] In some embodiments, as shown in FIG. 7, if the obstacle type is the through type, the filtering conditions of the obstacle include that the upper boundary 201 height of the passable area of the obstacle is greater than the height of the carrying device, the lower boundary 202 height of the passable area of the obstacle is less than the chassis height of the carrying device, and the width of the passable area of the obstacle is greater than the width of the carrying device.

[0140] The through type obstacle means that the carrying device passes through the middle of the passable area of the obstacle, so it is necessary to see whether the height and width of the passable area of the obstacle are greater than the height and width of the obstacle. If so, the carrying device can pass the obstacle, and the obstacle can be filtered out. Since in the height direction, it is also affected by the chassis height of the carrying device, the filtering conditions are set as follows: the upper boundary height of the passable area of the obstacle is greater than the height of the carrying device, the lower boundary height of the passable area of the obstacle is less than the chassis height of the carrying device, and the width of the passable area of the obstacle is greater than the width of the carrying device. If the passable area of the obstacle meets the above conditions, the carrying device can pass the obstacle. When generating two-dimensional data based on the filtered obstacle data, the two-dimensional data will not indicate that the filtered obstacle exists in the target scene.

[0141] Some embodiments of the present application provide a visual system, which includes a microprocessor and the binocular camera 300 involved in the above-mentioned embodiments. Since the binocular camera 300 can directly output obstacle data, the rear-end application device does not need complex algorithms, and a small-power microprocessor can be directly connected. More specifically, the microprocessor can be a single-chip microcomputer.

[0142] FIGS. 8 and 9 show schematic diagrams of the intelligent wheelchair provided by an embodiment of the present application. As shown in FIGS. 8 and 9, the intelligent wheelchair provided by the embodiment includes seat 12, backrest 13, foot pedal 14 and other components arranged above the skeleton 11. The user can sit on the seat 12, lean back on the backrest 13, and place the feet on the foot pedal 14, thereby ensuring the comfort of use. The wheels 15, chassis 16 and battery 17 are also installed below the skeleton 11.

[0143] The backrest 13 is also connected with an armrest assembly, which includes a left armrest 100 and a right armrest 200 arranged oppositely. The left armrest 100 is provided with an interactive screen 110 and a binocular camera 300, and the right armrest 200 is provided with a rocker 220 and a control panel 210, which is arranged with buttons.

[0144] As shown in FIG. 10, the end of the left armrest 100 includes a housing and the binocular camera 300 involved in the above embodiment, which is located in the housing for collecting obstacle data at the height of the armrest.

[0145] In some embodiments, the end of the left armrest 100 also includes a display screen 110, which is located in the housing for the user to operate the display screen when riding a vehicle or a wheelchair provided with the armrest structure.

[0146] In some embodiments, a video camera 500 is also installed on the binocular camera 300, which is called when making a video call with an external terminal device.

[0147] In some embodiments, the housing includes an upper housing structure 410 and a lower housing structure 420, the upper housing structure is provided with a first cavity 421, and the lower housing structure is provided with a second cavity 422, the display screen 110 is installed in the first cavity 421, and the binocular camera 300 is installed in the second cavity 422, when the upper housing structure 410 and the lower housing structure 420 cooperate, the first cavity 421 is located above the second cavity 422. In this way, on the one hand, the binocular camera can collect environmental information in front, and on the other hand, the user can easily operate the display screen. The display screen is also used to display the state and speed of the intelligent wheelchair, and to receive user input, such as mode setting of the intelligent wheelchair.

[0148] In some embodiments, the lower housing structure 420 also includes a protective back cover 423, which is located on the second cavity 422 and is used to protect the binocular camera.

[0149] In some embodiments, a display screen pad 430 is provided in the display screen housing, located between the display screen and the display screen cavity wall, for supporting the display screen.

[0150] Since the intelligent wheelchair is relatively slow, it does not need to use more accurate point cloud data for automatic navigation, and two-dimensional data representing the position of the obstacle can be used for automatic navigation. Therefore, the binocular camera can be used to replace the laser radar to reduce the cost of the intelligent wheelchair.

[0151] An embodiment of the present application provides a data processing chip 320, which includes a memory and a processor.

[0152] The memory is used to store computer execution instructions executable by the processor.

[0153] The processor implements each step in the method in the above embodiment when executing the computer-executable instructions. Details can be referred to the related description in the foregoing method embodiment.

[0154] Optionally, the above memory can be independent or integrated with the processor. When the memory is independently arranged, the data processing chip 320 further includes a bus for connecting the memory and the processor.

[0155] The embodiment of the present application further provides a computer readable storage medium, and the computer readable storage medium stores computer-executable instructions. When the processor executes the computer-executable instructions, each step in the method in the above embodiment is implemented.

[0156] The embodiment of the present application further provides a computer program product, and the computer program product includes computer-executable instructions. When the processor executes the computer-executable instructions, each step in the method in the above embodiment is implemented.

[0157] Other embodiments of the application will be apparent to those skilled in the art from consideration of the specification and practice of the application disclosed herein. The specification and examples given are exemplary only and the true scope and spirit of the application is indicated by the following claims. It will be appreciated by persons skilled in the art that numerous variations and / or modifications can be made to the application as described above without departing from the scope or spirit of the application. It is intended that all such variations and / or modifications be included within the scope of the application. The specification and examples given are exemplary only and the true scope and spirit of the application is indicated by the following claims.

[0158] It should be understood that the application is not limited to the precise construction that has been described above and illustrated in the accompanying drawings and that various modifications and changes can be made by those skilled in the art without departing from the scope of the application. The scope of the application is limited only by the claims that follow.

Claims

1. A binocular camera, characterized in that, The method comprises: a binocular camera for collecting a target scene and outputting parallax image data; a data processing chip electrically connected to the binocular camera, for processing the parallax image data to obtain a point cloud, and converting the point cloud into two-dimensional data for output, the two-dimensional data being used to represent the position of an obstacle in the target scene.

2. The binocular camera of claim 1, wherein, The two-dimensional data comprises a plurality of data points in polar coordinates; the plurality of data points in polar coordinates are used to represent the position of an obstacle.

3. The binocular camera of claim 1, wherein, The two-dimensional data further comprises a radar chart; the radar chart comprises a plurality of rays at different angles, the length of the ray being used to represent whether there is an obstacle, and the length of the ray and the angle of the ray being used to represent the position of the obstacle when there is an obstacle.

4. The binocular camera according to any one of claims 1 to 3, characterized in that, The binocular camera further receives external parameters, the external parameters being used to set an obstacle filtering condition, and the two-dimensional data not containing an obstacle satisfying the obstacle filtering condition.

5. The binocular camera of claim 4, wherein, The external parameters comprise the size of a mounting device mounting the binocular camera and / or the size of a passable obstacle.

6. The binocular camera according to any one of claims 1 to 3, wherein, The binocular camera further comprises a communication module; The communication module is electrically connected to the data processing chip, and the communication module is used to connect with an external microprocessor.

7. The binocular camera according to any one of claims 1 to 3, wherein, The binocular camera further comprises a light supplementing lamp; the light supplementing lamp provides light supplementing.

8. The binocular camera of claim 6, wherein, The binocular camera further comprises a printed circuit board; The binocular camera, the data processing chip and the communication module are located on the printed circuit board.

9. The binocular camera according to any one of claims 1 to 3, wherein, The data processing chip is a chip without AI function.

10. A vision system characterized by, The visual system comprises a microprocessor and the binocular camera as claimed in any one of claims 1 to 9.

11. The vision system of claim 10, wherein, The microprocessor comprises a single-chip microcomputer.

12. A smart wheelchair, characterized by The visual system as claimed in claim 10, the visual system comprising a seat, a backrest, an armrest and the binocular camera, the armrest being provided with the binocular camera at the end of the armrest.

13. The intelligent wheelchair of claim 12, wherein, The end of the armrest comprises a housing and the binocular camera, the binocular camera being located in the housing, and the end of the armrest further comprises a display screen, the display screen being located in the housing.

14. The intelligent wheelchair of claim 13, wherein, The housing comprises an upper housing structure and a lower housing structure; The upper housing structure is provided with a first cavity, and the lower housing structure is provided with a second cavity, the first cavity being used to mount the display screen, and the second cavity being used to mount the binocular camera; When the upper housing structure and the lower housing structure are matched, the first cavity is located above the second cavity.

15. A data processing method of a binocular camera, characterized by, The binocular camera comprises a binocular camera, and the method comprises: acquiring parallax image data output by the binocular camera; processing the parallax image data to obtain a point cloud, and converting the point cloud into two-dimensional data, the two-dimensional data being used to represent the position of an obstacle in a target scene.

16. The data processing method according to claim 15, characterized in that, Processing the parallax image data to obtain a point cloud, and converting the point cloud into two-dimensional data, specifically comprises: processing the parallax image data using a preset algorithm to obtain an obstacle data; projecting the obstacle data to obtain plane data of the obstacle, and generating the two-dimensional data according to the plane data of the obstacle.

17. The data processing method of claim 15, wherein, After processing the parallax image data to obtain a point cloud, and converting the point cloud into two-dimensional data, the method further comprises: filtering the obstacle data using a preset obstacle filtering condition; Accordingly, the obstacle data is projected to obtain the planar data of the obstacle, specifically including: The filtered obstacle data is projected to obtain the planar data of the obstacle.

18. The data processing method of claim 15, wherein, One obstacle type corresponds to one obstacle filtering condition, and each obstacle data includes obstacle three-dimensional data and obstacle type; Accordingly, the obstacle data is filtered using a preset obstacle filtering condition, specifically including: For each obstacle, if the three-dimensional data of the obstacle meets the filtering condition corresponding to the obstacle type, the three-dimensional data of the obstacle is deleted to obtain filtered obstacle data.

19. The data processing method of claim 18, wherein, If the obstacle type is a pit type, the filtering condition of the obstacle includes that the width of the obstacle along the forward direction of the carrying device is less than a preset width threshold; The preset width threshold is determined according to the wheel size of the carrying device.

20. The data processing method of claim 18, wherein, If the obstacle type is a protrusion type, the filtering condition of the obstacle includes that the height of the obstacle is less than a preset height threshold, and the preset height threshold is determined according to the chassis height of the carrying device.

21. The data processing method of claim 18, wherein, If the obstacle type is a pass-through type, the filtering condition of the obstacle includes that the upper boundary height of the passable area of the obstacle is greater than the height of the carrying device, the lower boundary height of the passable area of the obstacle is less than the chassis height of the carrying device, and the width of the passable area of the obstacle is greater than the width of the carrying device.

22. The data processing method of claim 16, wherein, The two-dimensional data is generated according to the planar data of the obstacle, specifically including: The profile data of the obstacle is generated according to the planar data of the obstacle; The intersection points of rays at different angles and the profile data of the obstacle are calculated to obtain a plurality of data points in the polar coordinate system.

Images