Vehicle information acquisition method and device based on visual system and laser radar
By combining the vehicle information acquisition method of vision system and lidar, and integrating environmental status and image data, the problem of identifying basic vehicle information in coal transportation is solved, and the accurate identification of vehicle information and the improvement of loading efficiency is achieved.
Patent Information
- Application Number
- CN202411481238.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-22
- Publication Date
- 2025-02-14
AI Technical Summary
The development of unmanned chemical loading technology in the field of coal transportation is limited by the difficulty of identifying basic information such as vehicle exterior profile, carriage profile, and vehicle loading tonnage, resulting in low working efficiency, high environmental noise, high dust and long loading cycle.
The vehicle information acquisition method based on the vision system and lidar is adopted to identify the vehicle environment state and image data through the vision system. The lidar collects point cloud data and combines the weight value to achieve accurate identification of the basic vehicle information.
It realizes accurate identification of vehicle information, solves the problem of difficulty in identifying visual systems in harsh environments, and overcomes the limitations of the use of lidar or visual systems alone, improving loading efficiency and environmental safety.
Smart Images
Figure CN119439184A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the fields of intelligent mines and image processing technology, and in particular to a method and device for acquiring vehicle information based on a visual system and a laser radar. Background Art
[0002] The development of unmanned loading technology, which accounts for a large proportion in the coal transportation field, is restricted by the recognition of basic information such as the vehicle's outer contour, the car body contour, and the vehicle's loading tonnage, and its development is slow. The coal loading system generally uses manual visual identification of the vehicle's axle number and predicts the vehicle's pre-loaded quantity to identify the vehicle's basic information, realizing semi-automatic loading under manual or human intervention. The work efficiency is low, and the working environment of the loaders is noisy and dusty, and the loading cycle is long. Summary of the invention
[0003] The present application provides a method and device for acquiring vehicle information based on a visual system and a laser radar.
[0004] The first purpose of this application is to propose a vehicle information acquisition method based on a visual system and a lidar, so as to realize the fusion of the lidar and the visual system, so as to enable accurate identification of vehicle information.
[0005] The second purpose of this application is to propose a vehicle information acquisition device based on a visual system and a lidar.
[0006] The third objective of the present application is to provide an electronic device.
[0007] A fourth objective of the present application is to provide a computer-readable storage medium.
[0008] To achieve the above-mentioned purpose, the first aspect of the present application proposes a vehicle information acquisition method based on a visual system and a laser radar, including: identifying the environmental state of the vehicle's current environment based on the visual system to obtain environmental state parameters; acquiring and identifying images of the vehicle based on the visual system to obtain first basic information of the vehicle; acquiring and identifying point cloud data of the vehicle based on the laser radar to obtain second basic information of the vehicle; determining, based on the configuration information, the first type of basic information that needs to be fused and the second type of basic information that does not need to be fused in the basic information of the vehicle, and determining, based on the environmental state parameters, the respective weight values of the visual system and the laser radar corresponding to the first type of basic information; performing weighted fusion of the first type of basic information in the first basic information and the second basic information based on the respective weight values of the visual system and the laser radar to obtain the first type of fused basic information of the vehicle; determining the second type of basic information from the first basic information or the second basic information; and obtaining the target basic information of the vehicle based on the first type of fused basic information and the second type of basic information.
[0009] To achieve the above-mentioned purpose, the second embodiment of the present application proposes a vehicle information acquisition device based on a visual system and a laser radar, including: a first recognition module, used to identify the environmental state of the current environment of the vehicle based on the visual system, and obtain environmental state parameters; a second recognition module, used to collect and identify the image of the vehicle based on the visual system, and obtain the first basic information of the vehicle; a third recognition module, used to collect and identify point cloud data of the vehicle based on the laser radar, and obtain the second basic information of the vehicle; a first determination module, used to determine the first type of basic information that needs to be fused and the second type of basic information that does not need to be fused in the basic information of the vehicle according to the configuration information, and determine the respective weight values of the visual system and the laser radar corresponding to the first type of basic information according to the environmental state parameters; a fusion module, used to weightedly fuse the first type of basic information in the first basic information and the second basic information based on the respective weight values of the visual system and the laser radar, and obtain the first type of fused basic information of the vehicle; a second determination module, used to determine the second type of basic information from the first basic information or the second basic information; a third determination module, used to obtain the target basic information of the vehicle according to the first type of fused basic information and the second type of basic information.
[0010] To achieve the above-mentioned purpose, the third aspect embodiment of the present application proposes an electronic device, comprising: a processor; and a memory communicatively connected to the processor; the memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory, so that the processor can execute the social media information generation method described in the first aspect embodiment above.
[0011] To achieve the above-mentioned purpose, the fourth aspect of the present application proposes a computer-readable storage medium having a computer program stored thereon, wherein the computer instructions are used to enable the computer to execute the social media information generation method described in the above-mentioned first aspect of the embodiment.
[0012] The vehicle information acquisition method and device based on the visual system and laser radar provided in the present application, identify the environmental state parameters of the vehicle's current environment and the first basic information of the vehicle through the visual system, and identify the second basic information of the vehicle based on the laser radar. By determining the first type of basic information and the second type of basic information from the basic information of the vehicle, and determining the weight values of the visual system and the laser radar based on the environmental state parameters, the first type of basic information and the second type of basic information are fused according to the weight values to obtain the first type of fused basic information of the vehicle. Furthermore, based on the first type of fused basic information and the second type of basic information, the target basic information of the vehicle can be obtained, and the fusion of the laser radar and the visual system can be realized to obtain accurate vehicle information. The problem that the visual system cannot accurately identify vehicle information in harsh environments and cannot effectively provide basic vehicle information is solved, and the limitation of using laser radar or visual system alone is solved, and the accurate identification of vehicle information is realized.
[0013] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS
[0014] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:
[0015] Figure 1 A flowchart of a method for acquiring vehicle information based on a visual system and a laser radar provided in an embodiment of the present application;
[0016] Figure 2 A flowchart of another method for acquiring vehicle information based on a visual system and a laser radar provided in an embodiment of the present application;
[0017] Figure 3 A flowchart of another method for acquiring vehicle information based on a visual system and a laser radar provided in an embodiment of the present application;
[0018] Figure 4a A schematic diagram of the front layout of the visual system and laser radar provided in an embodiment of the present application;
[0019] Figure 4b A schematic diagram of the left side layout of the visual system and laser radar provided in an embodiment of the present application;
[0020] Figure 4c A schematic diagram of the right side layout of the visual system and laser radar provided in an embodiment of the present application;
[0021] Figure 5 A schematic diagram of a process for obtaining vehicle information provided in an embodiment of the present application;
[0022] Figure 6 A schematic diagram of the structure of a vehicle information acquisition device based on a visual system and a laser radar provided in an embodiment of the present application. DETAILED DESCRIPTION
[0023] Embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.
[0024] The following describes a method and device for acquiring vehicle information based on a visual system and a laser radar according to an embodiment of the present application with reference to the accompanying drawings.
[0025] Figure 1 is a flow chart of a method for acquiring vehicle information based on a visual system and a laser radar according to an exemplary embodiment. Figure 1 As shown, the vehicle information acquisition method based on the visual system and laser radar in the embodiment of the present application includes but is not limited to the following steps:
[0026] S101, identifying the environmental state of the vehicle's current environment based on the visual system and obtaining environmental state parameters.
[0027] It should be noted that the execution subject of the vehicle information acquisition method based on the visual system and laser radar provided in the embodiment of the present application is an electronic device, which may be a terminal device. Optionally, the terminal device may be a mobile electronic device or a non-mobile electronic device. Exemplarily, the mobile electronic device may be a mobile phone, a tablet computer, a laptop computer, a PDA, an in-vehicle electronic device, a wearable device, an ultra-mobile personal computer (UMPC), a netbook or a personal digital assistant (PDA), etc., and the non-mobile electronic device may be a personal computer (PC), etc. The embodiment of the present application is not specifically limited.
[0028] In some implementations, the visual system may be based on an image acquisition device, which may be a camera, that is, a visual system may be a camera. Optionally, multiple visual systems may be used to identify the environmental state of the vehicle's current environment.
[0029] In some implementations, the visual system may collect an environmental image of the vehicle's current environment, and identify environmental state parameters in the environmental image to determine the current environmental state and obtain environmental state parameters. The environmental state parameters may be water vapor, fog, high dust, etc. The environmental state may be divided into a harsh environment and a non-harsh environment.
[0030] Optionally, based on the features of environmental state parameters such as water vapor, fog, high dust, etc. presented in the image, it is possible to identify whether environmental state parameters exist in the environmental image. If so, the environmental state and environmental state parameters can be determined.
[0031] By way of example, if the proportion of high dust in the environmental state parameters is greater than a set threshold, it is determined to be in a harsh environment; if the proportion of high dust in the environmental state parameters is less than or equal to the set threshold, it is determined to be in a non-harsh environment.
[0032] S102, performing image acquisition and recognition on the vehicle based on the visual system to obtain first basic information of the vehicle.
[0033] In some implementations, the visual system can collect images and identify the vehicle after the vehicle enters the identification area. Optionally, by judging whether the proportion of the vehicle in the identification area reaches a set value, if the proportion reaches the set value, it is determined that the vehicle has entered the identification area. The visual system collects images of the vehicle in the identification area and determines the proportion of the vehicle in the image. If the proportion reaches the set value, it is determined that the vehicle has entered the identification area, and collects and identifies the image of the vehicle to obtain the first basic information.
[0034] Optionally, the first basic information of the vehicle includes but is not limited to: first outer contour information of the vehicle, first compartment size information, license plate information and first load information. The first outer contour information refers to the length, width and height of the outer contour of the vehicle, and the first compartment size information refers to the length, width and height of the compartment.
[0035] That is, the visual system can identify the outer contour and the compartment contour of the vehicle in the image to obtain the first outer contour information and the first compartment size information of the vehicle. The visual system can determine the license plate information and the first load information by identifying the license plate and the key load text of the vehicle body in the image.
[0036] S103, collecting and identifying point cloud data of the vehicle based on the laser radar to obtain second basic information of the vehicle.
[0037] In some implementations, the point cloud data of the vehicle can be collected based on the laser radar, and the point cloud data can be identified to obtain the second basic information of the vehicle. Optionally, the second basic information of the vehicle includes but is not limited to: the second outer contour information of the vehicle, the second compartment size information, the license plate information, and the second load information. Among them, the second outer contour information refers to the length, width, and height of the outer contour of the vehicle, and the second compartment size information refers to the length, width, and height of the compartment.
[0038] In some implementations, after the laser radar collects point cloud data of the vehicle, the vehicle can be modeled based on the point cloud data to obtain the second outer contour information and second vehicle compartment size information of the vehicle, and based on the second outer contour information and the second vehicle compartment size information, the second load information of the vehicle can be predicted.
[0039] Optionally, the vehicle coordinate system in the point cloud data can be converted into a world coordinate system, and feature extraction and segmentation can be performed on the point cloud data to obtain vehicle feature data and segmentation data, such as vehicle body, wheels, windows, etc. The vehicle is modeled based on the world coordinate system, vehicle feature data and segmentation data.
[0040] S104, determining the first category of basic information that needs to be fused and the second category of basic information that does not need to be fused in the basic information of the vehicle according to the configuration information, and determining the respective weight values of the visual system and the laser radar corresponding to the first category of basic information according to the environmental state parameters.
[0041] In some implementations, the configuration information indicates the information type of different information in the basic information, and the information type includes: the first type of basic information that needs to be fused and the second type of basic information that does not need to be fused. Optionally, the information type of different information can be preset, and the configuration information is generated according to the set information type. For example, the license plate information is the second type of basic information, and the first outer contour information and the first compartment size information are the first type of basic information. In other words, the license plate information does not need to be fused, but the first outer contour information and the first compartment size information need to be fused.
[0042] Optionally, the basic information of the vehicle is first basic information and second basic information.
[0043] That is to say, the first type of basic information in the basic information of the vehicle refers to the first type of basic information in the first basic information and the second basic information. It can be determined that the first type of basic information in the first basic information includes: the first outer contour information, the first compartment size information and the first load information of the vehicle. The first type of basic information in the second basic information includes: the second outer contour information, the second compartment size information and the second load information. The second type of basic information in the basic information of the vehicle is the license plate information.
[0044] Furthermore, weight values can be assigned to the visual system and the laser radar corresponding to the first type of basic information according to the environmental state parameters. If the environmental state parameters indicate that the vehicle is currently in a harsh environment, a smaller weight value can be assigned to the visual system and a larger weight value can be assigned to the laser radar. In other words, when in a harsh environment, the weight value of the laser radar is greater than the weight value of the visual system.
[0045] If the environmental state parameter indicates that the vehicle is currently in a non-harsh environment, a larger weight value can be assigned to the visual system and a smaller weight value can be assigned to the lidar. In other words, when in a non-harsh environment, the weight value of the visual system is greater than the weight value of the lidar.
[0046] S105, based on the respective weight values of the visual system and the laser radar, weighted fusion is performed on the first basic information and the first type of basic information in the second basic information to obtain the first type of fused basic information of the vehicle.
[0047] In some implementations, the weight value of the visual system is the weight value of the first basic information, and the weight value of the laser radar is the weight value of the second basic information. According to the weight values of the first basic information and the second basic information, the first type of basic information in the first basic information and the second basic information can be weighted fused to obtain the first type of fused basic information of the vehicle.
[0048] S106, determining the second type of basic information from the first basic information or the second basic information.
[0049] Optionally, the second type of basic information may be determined from the first basic information or the second basic information based on the configuration information. For example, the information type in the first basic information is determined based on the configuration information, and the license plate information is determined as the second type of basic information based on the information type.
[0050] S107, obtaining target basic information of the vehicle according to the first type of fused basic information and the second type of basic information.
[0051] Optionally, the target basic information of the vehicle can be obtained by splicing the first type of fusion basic information and the second type of basic information. Optionally, the target basic information can be obtained by extracting feature data of the first type of fusion basic information and the second type of basic information and splicing the feature data. The target basic information includes but is not limited to: target outer contour information of the vehicle, target compartment size information, license plate information and target load information.
[0052] In some implementations, after obtaining the target basic information of the vehicle, the vehicle can be automatically loaded based on the target basic information. During the loading process, target load information, target outer contour information, target compartment size information and vehicle axle number information are required, and the license plate information is used as control information during the loading process.
[0053] Optionally, a vehicle loading control signal may be generated based on the target information of the vehicle, so that the vehicle is automatically loaded according to the control signal.
[0054] As an example, the target information of the vehicle is obtained by parsing the control signal. In the field of coal transportation, the target load information of the vehicle is the font sprayed on the door side of the front of the vehicle, which shows the rated load number of the vehicle and the number of passengers, and is used to control the overall loading weight; the target outer contour information is used to check whether the length, width and height data of the vehicle match the rated load of the front of the vehicle and whether it can load the rated load of coal. It can also be used to distinguish the relative position of the front of the vehicle and the carriage, whether the carriage can be matched under the coal discharge port, and whether the vehicle direction matches the coal discharge position; the target carriage size information is used to check whether the vehicle can load the rated weight of coal, and dynamically monitor the outer contour of the coal during the coal discharge process to see whether the coal is scattered or overflowed, and whether the loading weight matches the space; the vehicle axle number information is the number of axles of the whole vehicle including the front of the vehicle, which is used to estimate the overall length of the vehicle, the relative position of the front of the vehicle and the carriage, reverse check the basic parameters of the carriage, and determine the relative position of the vehicle relative to the coal discharge port during the loading process.
[0055] In the vehicle information acquisition method based on the visual system and the laser radar provided in the embodiment of the present application, the environmental state parameters of the current environment of the vehicle and the first basic information of the vehicle are identified by the visual system, and the second basic information of the vehicle is identified based on the laser radar. By determining the first type of basic information and the second type of basic information from the basic information of the vehicle, and according to the environmental state parameters, the weight values of the visual system and the laser radar are determined, so as to fuse the first type of basic information and the second type of basic information according to the weight values, and obtain the first type of fused basic information of the vehicle. Furthermore, based on the first type of fused basic information and the second type of basic information, the target basic information of the vehicle can be obtained, and the fusion of the laser radar and the visual system can be realized to obtain accurate vehicle information. The problem that the visual system cannot accurately identify vehicle information in harsh environments and cannot effectively provide basic vehicle information is solved, and the limitation of using the laser radar or the visual system alone is solved, and the accurate identification of vehicle information is realized.
[0056] Figure 2 is a flow chart of a method for acquiring vehicle information based on a visual system and a laser radar according to an exemplary embodiment. Figure 2 As shown, the vehicle information acquisition method based on the visual system and laser radar in the embodiment of the present application includes but is not limited to the following steps:
[0057] S201, identifying the environmental state of the vehicle's current environment based on the visual system and obtaining environmental state parameters.
[0058] In the embodiment of the present application, the implementation method of step S201 can be implemented by any method in the embodiments of the present application, which is not limited here and will not be repeated.
[0059] S202, collecting images of the vehicle based on a visual system, identifying the collected images of the vehicle, and obtaining a body area image, a license plate area image, and a door area image.
[0060] In some implementations, the number of vision systems and laser radars is N, and the vision systems and laser radars are symmetrically deployed, so that multiple vision systems and laser radars can collect images and point cloud data of the vehicle. The image of the vehicle is obtained by matching feature points of the images collected by the multiple vision systems and fusing the images collected by the multiple vision systems according to the first feature point matching result.
[0061] Optionally, feature points may be extracted from images captured by multiple visual systems as first feature points, and then feature point matching is performed on the first feature points to obtain first feature point matching results.
[0062] Furthermore, by dividing the vehicle into regions, images of different regions of the vehicle can be identified from the image of the vehicle, and then the first basic information can be obtained based on the images of the different regions. Optionally, the vehicle can be divided into a body region, a license plate region, and a door region, and the body region image, the license plate region image, and the door region image can be identified from the image of the vehicle.
[0063] S203, acquiring first outer contour information and first vehicle compartment size information of the vehicle according to the vehicle body area image.
[0064] In some implementations, edge features of the vehicle contour can be determined by performing edge detection on the vehicle body area image, and first outer contour information of the vehicle, that is, the length, width and height of the vehicle outer contour, can be extracted from the edge features.
[0065] In some implementations, edge features of the extracted vehicle contour may be identified to determine the compartment area, and the compartment area may be measured to obtain first compartment size information, that is, the length, width, and height of the compartment. Optionally, the first compartment size information may be measured using image processing technology.
[0066] S204, performing optical character recognition on the license plate area image to obtain the vehicle license plate information.
[0067] S205, performing optical character recognition on the door area image to obtain first load information of the vehicle.
[0068] In some implementations, optical character recognition is performed on the license plate area image and the door area image respectively to obtain the vehicle license plate information and the first load information of the vehicle. That is, the first basic information of the vehicle includes the first outer contour information of the vehicle, the first compartment size information, the first load information and the license plate information.
[0069] In some implementations, before performing image acquisition and point cloud data acquisition on the vehicle, it may be determined whether the vehicle has entered the identification area, and subsequent operations may be performed on the vehicle after entering the identification area. By monitoring whether the vehicle has entered the identification area, in response to the vehicle entering the identification area, image acquisition and identification of the vehicle based on the visual system is started to obtain the first basic information of the vehicle and subsequent operations.
[0070] Optionally, an image of the vehicle in the identification area can be collected based on a visual system, and in response to the proportion of the vehicle in the image reaching a set value, it is determined that the vehicle has entered the identification area. The pressure distribution of the vehicle in the identification area can also be obtained, and in response to the pressure distribution diagram of the vehicle in the identification area meeting the set conditions, it is determined that the vehicle has entered the identification area. The distance between the vehicle and the presence monitoring device in the identification area can also be obtained, and in response to the distance being less than or equal to the set value, it is determined that the vehicle has entered the identification area.
[0071] S206, collecting and identifying point cloud data of the vehicle based on the laser radar to obtain second basic information of the vehicle.
[0072] In the embodiment of the present application, the implementation method of step S206 can be implemented by any method in the embodiments of the present application, which is not limited here and will not be repeated.
[0073] S207: Determine, according to the configuration information, the first type of basic information that needs to be integrated and the second type of basic information that does not need to be integrated in the basic information of the vehicle.
[0074] In the embodiment of the present application, the implementation method of step S207 can be implemented by any method in the embodiments of the present application, which is not limited here and will not be repeated.
[0075] S208, determining the respective weight values of the visual system and the laser radar corresponding to the first type of basic information according to the environmental state parameters.
[0076] In some implementations, a characterization parameter of environmental quality may be determined based on the environmental state parameter, wherein the characterization parameter of environmental quality is used to indicate environmental quality. Further, different candidate intervals are determined, each of which corresponds to a different weight value of the visual system and the laser radar.
[0077] That is to say, by obtaining the target interval of the environmental quality characterization parameter, and according to the target interval, querying the mapping relationship between the interval and the weight, the weight values of the visual system and the lidar are determined. For example, interval A is (0,3), the corresponding visual system weight value is 0.8, and the lidar weight value is 0.2; interval B is (4,6), the corresponding visual system weight value is 0.3, and the lidar weight value is 0.7. If the environmental quality characterization parameter is 5, it is determined to be in interval B, then the weight values of the visual system and the lidar are 0.3 and 0.7 respectively.
[0078] Optionally, the characterization parameters of environmental quality can be predicted based on the environmental state parameters, and the characterization parameters of environmental quality can also be mapped based on the environmental state parameters. For example, by using a large model or a mapping function, the environmental state parameters are input to obtain the characterization parameters of environmental quality.
[0079] In some implementations, the environmental state parameters include multiple different state parameters, and the conditions satisfied by each state parameter in the environmental state parameters are obtained to determine the candidate weight values of the visual system and the laser radar according to the conditions satisfied by each state parameter. For example, the state parameters include water vapor parameters, fog parameters, and dust parameters. If the water vapor parameters meet the conditions of a non-harsh environment, the fog parameters meet the conditions of a harsh environment, and the dust parameters meet the conditions of a harsh environment, then the candidate weight values of the visual system and the laser radar in a non-harsh environment and the candidate weight values of the visual system and the laser radar in a harsh environment can be determined.
[0080] Furthermore, the candidate weight values of the visual system and the laser radar are averaged to obtain the weight values of the visual system and the laser radar.
[0081] S209, based on the respective weight values of the visual system and the laser radar, weighted fusion is performed on the first basic information and the first type of basic information in the second basic information to obtain the first type of fused basic information of the vehicle.
[0082] In the embodiment of the present application, the implementation method of step S209 can be implemented by any method in the various embodiments of the present application, which is not limited here and will not be repeated.
[0083] S210: Determine the second type of basic information from the first basic information or the second basic information.
[0084] In the embodiment of the present application, the implementation method of step S210 can be implemented by any method in the embodiments of the present application, which is not limited here and will not be repeated.
[0085] S211, obtaining target basic information of the vehicle according to the first type of fused basic information and the second type of basic information.
[0086] In the embodiment of the present application, the implementation method of step S211 can be implemented by any method in the embodiments of the present application, which is not limited here and will not be repeated.
[0087] In the vehicle information acquisition method based on the visual system and the laser radar provided in the embodiment of the present application, the environmental state parameters of the current environment of the vehicle and the first basic information of the vehicle are identified by the visual system, and the second basic information of the vehicle is identified based on the laser radar. By determining the first type of basic information and the second type of basic information from the basic information of the vehicle, and according to the environmental state parameters, the weight values of the visual system and the laser radar are determined, so as to fuse the first type of basic information and the second type of basic information according to the weight values, and obtain the first type of fused basic information of the vehicle. Furthermore, based on the first type of fused basic information and the second type of basic information, the target basic information of the vehicle can be obtained, and the fusion of the laser radar and the visual system can be realized to obtain accurate vehicle information. The problem that the visual system cannot accurately identify vehicle information in harsh environments and cannot effectively provide basic vehicle information is solved, and the limitation of using the laser radar or the visual system alone is solved, and the accurate identification of vehicle information is realized.
[0088] Figure 3 is a flow chart of a method for acquiring vehicle information based on a visual system and a laser radar according to an exemplary embodiment. Figure 3 As shown, the vehicle information acquisition method based on the visual system and laser radar in the embodiment of the present application includes but is not limited to the following steps:
[0089] S301, identifying the environmental state of the vehicle's current environment based on the visual system and obtaining environmental state parameters.
[0090] In the embodiment of the present application, the implementation method of step S301 can be implemented by any method in the embodiments of the present application, which is not limited here and will not be repeated.
[0091] S302, collecting and identifying images of the vehicle based on a visual system to obtain first basic information of the vehicle.
[0092] In the embodiment of the present application, the implementation method of step S302 can be implemented by any method in the embodiments of the present application, which is not limited here and will not be repeated.
[0093] S303, collecting point cloud data of the vehicle based on the laser radar, modeling the vehicle based on the collected point cloud data in combination with the reference coordinate system, and extracting the second outer contour information and the second vehicle compartment size information of the vehicle from the vehicle model.
[0094] In some implementations, the number of vision systems and laser radars is N, and the vision systems and laser radars are symmetrically deployed, so that multiple vision systems and laser radars can collect images and point cloud data of the vehicle. The point cloud data collected by multiple laser radars is matched with feature points, and the point cloud data collected by multiple laser radars are fused according to the second feature point matching result to obtain the point cloud data of the vehicle.
[0095] Optionally, feature points may be extracted from point cloud data collected by multiple laser radars as second feature points, and then feature point matching is performed on the second feature points to obtain second feature point matching results.
[0096] Optionally, the vehicle coordinate system in the point cloud data can be converted into a world coordinate system, and feature extraction and segmentation can be performed on the point cloud data to obtain vehicle feature data and segmentation data, such as vehicle body, wheels, windows, etc. Based on the world coordinate system, vehicle feature data and segmentation data, the vehicle can be modeled to obtain a vehicle model.
[0097] Furthermore, by performing operations such as outer contour extraction and vehicle compartment measurement on the vehicle model, second outer contour information and second vehicle compartment size information of the vehicle can be obtained.
[0098] For example, Figure 4a The figure shows the front layout of the vision system and the laser radar, where the vision system is a camera. Figure 4a In the figure, 1 represents the quantitative warehouse, 2 represents the coal gate, 3 represents the vehicle, 4 represents the laser radar under the warehouse, 5 represents the camera under the warehouse, 6 represents the laser radar under the warehouse, 7 represents the camera under the warehouse, 8 represents the low-position laser radar, and 9 represents the low-position camera. In other words, there are 3 visual systems and 3 laser radars to identify the vehicle information.
[0099] Figure 4b The diagram shows the left side layout of the visual system and the laser radar, where the visual system is a camera. Figure 4b Among them, 1 represents the quantitative bin, 2 represents the coal discharge gate, 3 represents the vehicle, 4 represents the camera under the bin, 5 represents the laser radar under the bin, 6 represents the low-position camera, 7 represents the low-position laser radar, and 8 represents the key information text of the vehicle body.
[0100] Figure 4c The figure shows the layout diagram of the right side of the visual system and the laser radar, where the visual system is a camera. Figure 4c Among them, 1 represents the quantitative bin, 2 represents the coal discharge gate, 3 represents the vehicle, 4 represents the under-bin laser radar, 5 represents the under-bin camera, 6 represents the low-position camera, 7 represents the low-position laser radar, and 8 represents the key information text of the vehicle body.
[0101] S304: predicting the load of the vehicle according to the second outer contour information and the second vehicle compartment size information to obtain second load information of the vehicle.
[0102] Optionally, a prediction model may be pre-trained, and the trained prediction model may be used to predict the vehicle load in combination with the second outer contour information and the second compartment size information. By collecting sample data and using the sample data to train the model, the model can accurately predict the vehicle load.
[0103] That is, the second basic information of the vehicle includes the second outer contour information, the second compartment size information and the second load information of the vehicle.
[0104] S305, determining the first category of basic information that needs to be fused and the second category of basic information that does not need to be fused in the basic information of the vehicle according to the configuration information, and determining the respective weight values of the visual system and the laser radar corresponding to the first category of basic information according to the environmental state parameters.
[0105] In the embodiment of the present application, the implementation method of step S305 can be implemented by any method in the embodiments of the present application, which is not limited here and will not be repeated.
[0106] S306, based on the respective weight values of the visual system and the laser radar, weighted fusion is performed on the first basic information and the first type of basic information in the second basic information to obtain the first type of fused basic information of the vehicle.
[0107] In the embodiment of the present application, the implementation method of step S306 can be implemented by any method in the embodiments of the present application, which is not limited here and will not be repeated.
[0108] S307: Determine the second type of basic information from the first basic information or the second basic information.
[0109] In the embodiment of the present application, the implementation method of step S307 can be implemented by any method in the embodiments of the present application, which is not limited here and will not be repeated.
[0110] S308, obtaining target basic information of the vehicle according to the first type of fused basic information and the second type of basic information.
[0111] In the embodiment of the present application, the implementation method of step S308 can be implemented by any method in the embodiments of the present application, which is not limited here and will not be repeated.
[0112] In the vehicle information acquisition method based on the visual system and the laser radar provided in the embodiment of the present application, the environmental state parameters of the current environment of the vehicle and the first basic information of the vehicle are identified by the visual system, and the point cloud data of the vehicle is identified according to the laser radar, and the second basic information is obtained from the point cloud data. By determining the first type of basic information and the second type of basic information from the basic information of the vehicle, and determining the weight values of the visual system and the laser radar according to the environmental state parameters, the first type of basic information and the second type of basic information are fused according to the weight values to obtain the first type of fused basic information of the vehicle. Furthermore, based on the first type of fused basic information and the second type of basic information, the target basic information of the vehicle can be obtained, and the fusion of the laser radar and the visual system can be realized to obtain accurate vehicle information. The problem that the visual system cannot accurately identify vehicle information in harsh environments and cannot effectively provide basic vehicle information is solved, and the limitation of using the laser radar or the visual system alone is solved, and the accurate identification of vehicle information is realized.
[0113] Figure 5 The figure shows a flow chart of obtaining vehicle information. By determining that the vehicle enters the identification area, the visual system is used to identify the environment in which the vehicle is located, and the environmental state parameters are obtained. The weight values of the visual system and the laser radar are determined according to the environmental state parameters. The image of the vehicle is collected by the visual system, and contour recognition, text recognition and other operations are performed on the collected image to obtain the first outer contour information, first compartment size information, license plate information and first load information of the vehicle as the first basic information.
[0114] By using a laser radar to collect point cloud data of the vehicle, and performing vehicle modeling based on the point cloud data, a vehicle model is obtained, so as to extract the second outer contour information and the second vehicle compartment size information of the vehicle from the vehicle model, and use the second outer contour information and the second vehicle compartment size information to predict the load of the vehicle, so as to obtain the second load information of the vehicle, and use the second outer contour information, the second vehicle compartment size information, the license plate information and the second load information as the second basic information.
[0115] Furthermore, configuration information is obtained, and based on the configuration information, the first category of basic information that needs to be fused and the second category of basic information that does not need to be fused in the first basic information and the second basic information are determined, and the first category of basic information is weightedly fused according to the respective weight values of the visual system and the lidar to obtain the first category of fused basic information of the vehicle, and then the target basic information of the vehicle can be obtained based on the first category of fused basic information and the second category of basic information.
[0116] In order to implement the above embodiments, the present application also proposes a vehicle information acquisition device based on a visual system and a laser radar.
[0117] Figure 6A schematic diagram of the structure of a vehicle information acquisition device based on a visual system and a laser radar provided in an embodiment of the present application.
[0118] like Figure 6 As shown, the vehicle information acquisition device 600 based on the visual system and the laser radar includes:
[0119] The first recognition module 601 is used to recognize the environmental state of the vehicle's current environment based on the visual system and obtain environmental state parameters;
[0120] A second recognition module 602 is used to collect and recognize images of the vehicle based on the visual system to obtain first basic information of the vehicle;
[0121] The third identification module 603 is used to collect and identify point cloud data of the vehicle based on the laser radar to obtain the second basic information of the vehicle;
[0122] A first determination module 604 is used to determine, according to the configuration information, the first type of basic information that needs to be fused and the second type of basic information that does not need to be fused in the basic information of the vehicle, and determine, according to the environmental state parameter, the respective weight values of the visual system and the laser radar corresponding to the first type of basic information;
[0123] A fusion module 605 is used to perform weighted fusion on the first basic information and the first type of basic information in the second basic information based on the respective weight values of the visual system and the laser radar, so as to obtain the first type of fused basic information of the vehicle;
[0124] A second determining module 606, configured to determine the second-category basic information from the first basic information or the second basic information;
[0125] The third determination module 607 is used to obtain the target basic information of the vehicle according to the first type of fusion basic information and the second type of basic information.
[0126] In a possible implementation of an embodiment of the present application, the first determination module 604 is also used to: determine a characterization parameter of environmental quality based on the environmental state parameter; obtain a target interval in which the characterization parameter of environmental quality is located, and determine the respective weight values of the visual system and the lidar based on the mapping relationship between the target interval, the query interval and the weight.
[0127] In a possible implementation method of an embodiment of the present application, the first determination module 604 is also used to: obtain the conditions satisfied by each state parameter in the environmental state parameters; determine the candidate weight values of the visual system and the laser radar respectively according to the conditions satisfied by each state parameter; and average the candidate weight values of the visual system and the laser radar respectively to obtain the weight values of the visual system and the laser radar respectively.
[0128] In a possible implementation of the embodiment of the present application, the second recognition module 602 is further used to: recognize the collected image of the vehicle to obtain a body area image, a license plate area image and a door area image; obtain the first outer contour information and the first vehicle compartment size information of the vehicle based on the body area image; perform optical character recognition on the license plate area image to obtain the license plate information of the vehicle; perform optical character recognition on the door area image to obtain the first load information of the vehicle.
[0129] In a possible implementation of an embodiment of the present application, the third identification module 603 is also used to: perform vehicle modeling on the collected point cloud data in combination with a reference coordinate system, and extract the second outer contour information and second vehicle compartment size information of the vehicle from the vehicle model; predict the load of the vehicle based on the second outer contour information and the second vehicle compartment size information to obtain the second load information of the vehicle.
[0130] In a possible implementation of an embodiment of the present application, the device also includes: performing feature point matching on the images captured by the multiple visual systems, and fusing the images captured by the multiple visual systems based on a first feature point matching result to obtain an image of the vehicle; performing feature point matching on the point cloud data captured by the multiple lidars, and fusing the point cloud data captured by the multiple lidars based on a second feature point matching result to obtain point cloud data of the vehicle.
[0131] In a possible implementation of an embodiment of the present application, the second recognition module 602 is also used to: monitor whether the vehicle enters the recognition area, and in response to the vehicle entering the recognition area, start executing the image acquisition and recognition of the vehicle based on the visual system to obtain the first basic information of the vehicle and subsequent operations.
[0132] In the vehicle information acquisition device based on the visual system and the laser radar provided in the embodiment of the present application, the environmental state parameters of the current environment of the vehicle and the first basic information of the vehicle are identified by the visual system, and the second basic information of the vehicle is identified based on the laser radar. By determining the first type of basic information and the second type of basic information from the basic information of the vehicle, and determining the weight values of the visual system and the laser radar based on the environmental state parameters, the first type of basic information and the second type of basic information are fused according to the weight values to obtain the first type of fused basic information of the vehicle. Furthermore, based on the first type of fused basic information and the second type of basic information, the target basic information of the vehicle can be obtained, and the fusion of the laser radar and the visual system is realized to obtain accurate basic information of the vehicle. The problem that the visual system cannot accurately identify vehicle information in harsh environments and cannot effectively provide basic information of the vehicle is solved, and the limitation of using the laser radar or the visual system alone is solved, and the accurate identification of vehicle information is realized.
[0133] It should be noted that the aforementioned explanation of the embodiment of the vehicle information acquisition method based on a visual system and a laser radar is also applicable to the vehicle information acquisition device based on a visual system and a laser radar of this embodiment, and will not be repeated here.
[0134] In order to implement the above embodiments, the present application also proposes an electronic device, comprising: a processor, and a memory communicatively connected to the processor; the memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory to implement the method provided by the above embodiments.
[0135] In order to implement the above embodiments, the present application also proposes a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, they are used to implement the methods provided by the above embodiments.
[0136] The collection, storage, use, processing, transmission, provision and application of user personal information involved in this application are in compliance with relevant laws and regulations and do not violate public order and good morals.
[0137] It should be noted that personal information from users should be collected for legitimate and reasonable purposes and should not be shared or sold outside of these legitimate uses. In addition, such collection / sharing should be carried out after receiving the user's informed consent, including but not limited to notifying the user to read the user agreement / user notice and sign the agreement / authorization including authorization of relevant user information before the user uses the function. In addition, any necessary steps should be taken to protect and safeguard access to such personal information data and ensure that others who have access to personal information data comply with its privacy policy and procedures.
[0138] This application is expected to provide an implementation scheme for users to selectively block the use or access of personal information data. That is, this application is expected to provide hardware and / or software to prevent or block access to such personal information data. Once the personal information data is no longer needed, risks can be minimized by limiting data collection and deleting data. In addition, when applicable, personal identifiers are removed from such personal information to protect the privacy of users.
[0139] In the description of the aforementioned embodiments, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.
[0140] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of this application, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.
[0141] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, fragment or portion of code comprising one or more executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present application belong.
[0142] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute the instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purpose of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or processing in other suitable ways if necessary, and then stored in a computer memory.
[0143] It should be understood that the various parts of the present application can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.
[0144] A person skilled in the art may understand that all or part of the steps in the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.
[0145] In addition, each functional unit in each embodiment of the present application may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.
[0146] The storage medium mentioned above may be a read-only memory, a magnetic disk or an optical disk, etc. Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limiting the present application. A person of ordinary skill in the art may change, modify, replace and modify the above embodiments within the scope of the present application.
Claims
1. A vehicle information acquisition method based on a visual system and a laser radar, characterized in that: include: Identify the environmental state of the vehicle's current environment based on the visual system and obtain environmental state parameters; Capturing and identifying an image of the vehicle based on the visual system to obtain first basic information of the vehicle; Collecting and identifying point cloud data of the vehicle based on a laser radar to obtain second basic information of the vehicle; Determine, according to the configuration information, first-category basic information that needs to be fused and second-category basic information that does not need to be fused in the basic information of the vehicle, and determine, according to the environmental state parameters, respective weight values of the visual system and the laser radar corresponding to the first-category basic information; Based on the respective weight values of the visual system and the laser radar, weighted fusion is performed on the first basic information and the first type of basic information in the second basic information to obtain the first type of fused basic information of the vehicle; Determine the second category of basic information from the first basic information or the second basic information; The target basic information of the vehicle is obtained according to the first-category fusion basic information and the second-category basic information.
2. The method according to claim 1, characterized in that The determining, according to the environmental state parameter, respective weight values of the visual system and the laser radar corresponding to the first type of basic information includes: Determining a parameter representing environmental quality according to the environmental state parameter; The target interval of the characterization parameter of the environmental quality is obtained, and the weight values of the visual system and the laser radar are determined according to the mapping relationship between the target interval, the query interval and the weight.
3. The method according to claim 1, characterized in that The determining, according to the environmental state parameter, respective weight values of the visual system and the laser radar corresponding to the first type of basic information includes: Obtaining the conditions satisfied by each state parameter in the environmental state parameters; Determine candidate weight values for the visual system and the laser radar, respectively, according to the conditions satisfied by each of the state parameters; The candidate weight values of the visual system and the laser radar are averaged to obtain the weight values of the visual system and the laser radar.
4. The method according to any one of claims 1 to 3, characterized in that The first basic information of the vehicle includes first outer contour information, first compartment size information, first load information and license plate information of the vehicle. The first basic information of the vehicle is obtained by performing image acquisition and recognition on the vehicle based on the visual system, including: Identify the captured image of the vehicle to obtain a body area image, a license plate area image, and a door area image; Acquire first outer contour information and first compartment size information of the vehicle according to the vehicle body area image; Performing optical character recognition on the license plate area image to obtain the license plate information of the vehicle; Optical character recognition is performed on the door area image to obtain first load information of the vehicle.
5. The method according to any one of claims 1 to 3, characterized in that: The second basic information of the vehicle includes the second outer contour information, the second compartment size information and the second load information of the vehicle. The acquisition and recognition of point cloud data of the vehicle based on the laser radar to obtain the second basic information of the vehicle includes: In combination with a reference coordinate system, vehicle modeling is performed on the collected point cloud data, and second outer contour information and second compartment size information of the vehicle are extracted from the vehicle model; The load of the vehicle is predicted according to the second outer contour information and the second vehicle compartment size information to obtain second load information of the vehicle.
6. The method according to any one of claims 1 to 3, characterized in that The number of the visual systems and the number of the laser radars are both N, and the visual systems and the laser radars are symmetrically arranged. The method further includes: Performing feature point matching on the images collected by the multiple visual systems, and fusing the images collected by the multiple visual systems according to a first feature point matching result to obtain an image of the vehicle; Feature point matching is performed on the point cloud data collected by the multiple laser radars, and based on the second feature point matching result, the point cloud data collected by the multiple laser radars are fused to obtain the point cloud data of the vehicle.
7. The method according to any one of claims 1 to 3, characterized in that The performing image acquisition and recognition on the vehicle based on the visual system to obtain first basic information of the vehicle includes: Whether the vehicle enters the identification area is monitored, and in response to the vehicle entering the identification area, image acquisition and identification of the vehicle based on the visual system is started to obtain the first basic information of the vehicle and subsequent operations.
8. A vehicle information acquisition device based on a visual system and a laser radar, characterized in that: include: A first recognition module is used to recognize the environmental state of the vehicle's current environment based on a visual system and obtain environmental state parameters; A second recognition module, configured to collect and recognize an image of the vehicle based on the visual system to obtain first basic information of the vehicle; A third identification module is used to collect and identify point cloud data of the vehicle based on a laser radar to obtain second basic information of the vehicle; A first determination module is used to determine, according to the configuration information, first-category basic information that needs to be fused and second-category basic information that does not need to be fused in the basic information of the vehicle, and to determine, according to the environmental state parameters, respective weight values of the visual system and the laser radar corresponding to the first-category basic information; a fusion module, configured to perform weighted fusion on the first basic information and the first type of basic information in the second basic information based on respective weight values of the visual system and the laser radar, so as to obtain the first type of fused basic information of the vehicle; A second determining module, configured to determine the second-category basic information from the first basic information or the second basic information; The third determination module is used to obtain the target basic information of the vehicle according to the first type of fusion basic information and the second type of basic information.
9. An electronic device, characterized in that: include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory to implement the method according to any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 7 when executed by a processor.