Image recognition method and system, program product, electronic equipment and vehicle

Abstract

The invention relates to an image recognition method and system, a program product, electronic equipment and a vehicle, and the method comprises the steps: obtaining the environment data of the vehicle through a data obtaining module, and enabling the environment data to comprise target data; the target data is identified through the image identification unit to obtain the identification result, and the target data is identified through the image identification unit, so that the image identification unit can optimize the data acquisition effect according to the environment data, the quality of the target data is not influenced by the environment light, the quality of the target data is ensured, and therefore, the accuracy of the target data is improved. The detection efficiency of the code image in the target data can be improved, the code scanning recognition efficiency of the vehicle is improved, and effective code scanning of the vehicle is realized.

Description

Technical Field

[0001] The present application relates to the field of intelligent driving technology, and in particular to an image recognition method, system, program product, electronic device and vehicle. Background Art

[0002] The current mainstream mobile scanning method is mobile phone scanning. However, using the vehicle's onboard camera to scan the vehicle's QR code in a car scenario has the advantage of being more convenient and safer, because it avoids the tedious operation of users taking out their mobile phones to scan the code, allowing users to focus on driving.

[0003] In related technologies, code scanning is accomplished by detecting, displaying, and analyzing coded patterns (such as QR codes and barcodes) using images captured by vehicle-mounted cameras. However, this type of image capture based on vehicle-mounted cameras is easily affected by ambient lighting, which reduces the efficiency of code recognition and makes it difficult to effectively scan codes in vehicles. Summary of the Invention

[0004] The embodiments of the present application provide an image recognition method, system, program product, electronic device, and vehicle, which can realize effective code scanning of vehicles to at least partially solve the above-mentioned technical problems.

[0005] In order to achieve the above-mentioned object, according to a first aspect of the present application, there is provided an image recognition method, the method comprising:

[0006] Acquire environmental data of the vehicle through a data acquisition module, wherein the environmental data includes target data;

[0007] The target data is identified by an image recognition unit to obtain a recognition result.

[0008] According to a second aspect of the present application, there is provided an image recognition system, the system comprising:

[0009] A data acquisition module and a data identification unit, wherein the data acquisition module and the data identification unit are communicatively connected;

[0010] The data acquisition module is used to acquire the environmental data of the vehicle;

[0011] The data identification unit is used to receive and identify target data in the environmental data.

[0012] According to a third aspect of the present application, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the above-mentioned image recognition method is implemented.

[0013] According to a fourth aspect of the present application, a computer program product is provided, comprising a computer program, wherein the computer program implements the above-mentioned image recognition method when executed by a processor.

[0014] According to a fifth aspect of the present application, an electronic device is provided, comprising: a memory on which a computer program is stored; and a processor for executing the computer program in the memory to implement the above-mentioned image recognition method.

[0015] According to a sixth aspect of the present application, a vehicle is provided, comprising the above-mentioned image recognition system or electronic device or executing the image recognition method described in the first aspect.

[0016] The image recognition method, system, program product, electronic device and vehicle of the embodiments of the present application obtain environmental data of the vehicle through a data acquisition module, and the environmental data includes target data; the target data is recognized by an image recognition unit to obtain a recognition result. Since the target data in the environmental data is recognized by the image recognition unit, the image recognition unit can optimize the data acquisition effect according to the environmental data, so that the quality of the target data is not affected by the ambient light, thereby ensuring the quality of the target data. Therefore, it can improve the detection efficiency of the code image in the target data, improve the vehicle's scanning efficiency, and realize effective vehicle scanning.

[0017] Other features and advantages of the present application will be described in detail in the subsequent detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] To more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present application. Those skilled in the art can also derive other drawings based on these drawings without inventive effort.

[0019] In order to more completely understand the present application and its beneficial effects, the following description will be given in conjunction with the accompanying drawings, wherein the same drawing numbers represent the same parts in the following description.

[0020] Figure 1 is a schematic diagram of an image captured by a vehicle-mounted camera in a dark environment provided in some embodiments of the present application;

[0021] Figure 2 is a flowchart of an image recognition method provided in some embodiments of the present application;

[0022] Figure 3 is a schematic diagram of first image data collected when a vehicle is traveling at a relatively high speed, provided in some embodiments of the present application;

[0023] Figure 4This is a schematic diagram of the input and output of a vehicle code scanning system combining an image camera and an event camera provided in some embodiments of the present application;

[0024] Figure 5 is a schematic diagram of two frames of target event stream data and second image data provided in some embodiments of the present application;

[0025] Figure 6 This is a flowchart of scanning a code based on the first recognition result provided in some embodiments of the present application;

[0026] Figure 7 is a flowchart of scanning a code based on a second recognition result provided in some embodiments of the present application;

[0027] Figure 8 is a schematic structural diagram of an electronic device provided in some embodiments of the present application;

[0028] Figure 9 is a schematic diagram of a vehicle provided in some embodiments of the present application. DETAILED DESCRIPTION

[0029] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the embodiments described are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present application.

[0030] In the description of this application, the terms "first" and "second" are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly specify the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more features. In the description of this application, "plurality" means two or more, unless otherwise specifically defined.

[0031] In the description of this application, the word "for example" is used to mean "used as an example, illustration or illustration". Any embodiment described in this application as "for example" is not necessarily to be construed as being more preferred or advantageous than other embodiments. The following description is given to enable any person skilled in the art to implement and use the present invention. In the following description, details are listed for the purpose of explanation. It should be understood that a person of ordinary skill in the art can recognize that the present invention can be implemented without using these specific details. In other examples, well-known structures and processes are not elaborated in detail to avoid obscuring the description of the present invention with unnecessary details. Therefore, the present invention is not intended to be limited to the embodiments shown, but is consistent with the widest scope consistent with the principles and features disclosed in this application.

[0032] In related technologies, scanning is usually completed by detecting, displaying, and parsing coded patterns (such as QR codes, barcodes, etc.) through images collected by vehicle-mounted cameras. However, in poor lighting conditions, such as Figure 1 The figure below shows an image captured by an on-board camera in a low-light environment. When scanning a QR code in a dark environment, the on-board camera experiences underexposure, making it impossible to interpret the QR code content (the same applies to overexposure). Therefore, using an on-board camera such as a complementary metal-oxide semiconductor (CMOS) camera is not sufficient for actual code scanning needs.

[0033] In order to solve the above problems, an embodiment of the present application provides an image recognition method to obtain the environmental data of the vehicle; the target data in the environmental data is identified by the image recognition unit to obtain a recognition result. Since the target data in the environmental data is identified by the image recognition unit, the image recognition unit can optimize the data acquisition effect according to the environmental data, so that the quality of the target data is not affected by the ambient light, thereby ensuring the quality of the target data. Therefore, it can improve the detection efficiency of the code image in the target data, improve the vehicle's code scanning and recognition efficiency, and realize effective vehicle code scanning.

[0034] See also Figure 2 , provides an image recognition method, the method comprising:

[0035] Step S101: Acquire vehicle environment data through a data acquisition module.

[0036] Environmental data refers to data representing the vehicle itself and its surroundings, such as vehicle driving data, light intensity, and image data of the vehicle's surroundings, and includes target data. The target data includes an image of a code to be scanned, such as an image of a barcode or a two-dimensional barcode. In some embodiments, the target data is graphic code data, including at least one of a one-dimensional code, a two-dimensional code, a barcode, a matrix code, and a trademark code.

[0037] The data acquisition module is a module used to acquire environmental data of the vehicle, including but not limited to sensors or image acquisition devices.

[0038] Specifically, the data acquisition module acquires the environmental data of the vehicle, which includes target data, so that the image recognition unit can be optimized according to the environmental data to improve the image recognition effect.

[0039] Step S102: identifying the target data through an image recognition unit to obtain a recognition result.

[0040] The image recognition unit is an image processing unit configured on the vehicle for collecting environmental data of the vehicle and identifying target data in the environmental data. The recognition method can be at least one of an OpenCV-based method, a pyzbar-based method, or a Zbar-based method.

[0041] Among them, the specific implementation process of the method based on OpenCV can be: (1) Image preprocessing: grayscale, binarization and other preprocessing operations are performed on the input image to remove color information and noise, enhance the contrast of the QR code image, and facilitate subsequent detection. (2) Positioning pattern detection: use the cv2.QRCodeDetector class in OpenCV to detect the positioning pattern in the QR code image and determine the boundary and direction of the QR code. For example, create a QR code detector detector = cv2.QRCodeDetector(), and then call the detector.detectAndDecode(thresh) method, where thresh is the image after binarization. This method will return the recognition result, the point of the positioning pattern and other information. (3) Data decoding: decode the data in the QR code matrix according to the position and direction of the positioning pattern to extract the encoded information. If the recognition result returned by the detectAndDecode method is not empty, it means that there is a QR code in the image.

[0042] The recognition result refers to the result information obtained by performing code recognition on the target data, which includes a detection mark indicating whether the code image is detected and the detected code image.

[0043] Specifically, based on the current environmental state represented in the environmental data, an image acquisition device that matches the current environmental state can be selected to acquire data for image recognition, thereby ensuring that the data used for image recognition is not affected by the environment, improving the quality of the data, and thus improving the detection efficiency of code images in the target data, improving the vehicle's code scanning and recognition efficiency, and realizing effective vehicle code scanning.

[0044] In some embodiments, the identifying target data in the environmental data by the image recognition unit to obtain the identification result includes: calling the image recognition unit to collect the data to be identified according to the environmental data, and identifying the target data in the data to be identified to obtain the identification result.

[0045] The environmental data includes data reflecting the current ambient lighting state. For example, the ambient lighting state is normal, such as relatively bright or relatively dark; the ambient lighting state is bad, such as very bright or very dark (such as pitch black).

[0046] The image recognition unit is used to collect data to be recognized that needs to be subjected to code image recognition, that is, the data to be recognized contains target data.

[0047] Specifically, based on the data in the environmental data that can reflect the current environmental state, an image recognition unit that matches the current environmental state can be invoked to collect data to be recognized, and target data in the data to be recognized can be recognized to obtain a recognition result. It can be understood that in this embodiment, invoking the image recognition unit based on the environmental data ensures that the data to be recognized collected by the image recognition unit is not affected by ambient lighting, thereby improving the quality of the data to be recognized and further facilitating improved recognition efficiency of the target data.

[0048] In some embodiments, the data acquisition module includes an image camera and an event camera, and acquiring the environmental data of the vehicle through the data acquisition module includes: acquiring the environmental data through the image camera and generating first image data based on the environmental data; acquiring the environmental data through the event camera and generating event stream data based on the environmental data.

[0049] In some embodiments, if the environmental data is first environmental data, the image recognition unit is a camera; if the environmental data is second environmental data, the image recognition unit is a radar.

[0050] The first environmental data may indicate that the current lighting conditions are normal. The second environmental data may indicate that the current lighting conditions are poor. The camera may be an image camera or an event camera. The radar is a radar sensor, such as a lidar sensor.

[0051] Specifically, by selecting different image recognition units (such as cameras or radars) according to the type of environmental data (such as first environmental data or second environmental data), the acquisition method can be flexibly adjusted according to the needs of specific scenarios, thereby better adapting to different image recognition environments.

[0052] For example, in environments with good lighting conditions, using a camera to collect data to be identified can obtain high-resolution visual information; while in low-light or complex weather conditions, radar can provide more reliable depth and distance information. This can improve the efficiency of data collection and subsequent recognition.

[0053] Because radar sensors are not affected by light compared to image cameras, the collected point cloud data still has good quality even in poor lighting environments. Code recognition is performed on the point cloud data to ensure that the detected code image in the recognition result is of good quality, and the code image can be scanned efficiently, realizing effective code scanning of vehicles in poor lighting scenes and improving the user experience of the code scanning function.

[0054] Among them, the image camera can be a CMOS camera, and the event camera can be a dynamic vision sensor (DVS), an asynchronous time-based image sensor (ATIS), or a dynamic and active pixel vision sensor (DAVIS).

[0055] The first image data is image data captured by an image camera, for example, an RGB image, and the event stream data is data captured by an event camera.

[0056] The inventors found that the image camera has the following three defects in the scene of vehicle code scanning: First, motion blur will be generated, such as Figure 3The figure below is a schematic diagram of first image data captured while a vehicle is traveling at a relatively high speed. This first image data is an image of a QR code posted on a pillar next to the lane. Clearly, the first image data exhibits motion blur, making it impossible to parse the QR code content. Secondly, the image camera has a low dynamic range, and in extremely dim or bright lighting conditions, the information it captures is limited. Thirdly, the CMOS camera captures images at a constant frequency, which introduces a certain degree of latency, making it easy to miss targets. This makes it impossible for an in-vehicle code scanning system based solely on an image camera to effectively scan QR codes in poor lighting conditions and fast-moving vehicles.

[0057] Event cameras offer the following advantages in vehicle code scanning scenarios: First, they have high temporal resolution (with microsecond-level temporal resolution), which resists motion blur and avoids missing moving targets. Furthermore, they use asynchronous imaging, sensing only changes in brightness. Their imaging is independent of absolute ambient light intensity, but rather dependent on variations in ambient light intensity. This allows them to adapt to various lighting conditions and detect the light and dark blocks of a QR code or barcode. However, when the target (QR code or conditional code) is relatively stationary relative to the vehicle, or when the vehicle's speed is low or zero (such as when the vehicle is about to start), the event camera cannot detect the target. In this case, code recognition can be performed in conjunction with the first image data captured by the image camera to identify the code image.

[0058] like Figure 4 The figure shows an input and output diagram of the vehicle code scanning system in this embodiment. The vehicle code scanning system includes a camera, a radar and a domain control computing device. The camera and the radar can be installed on the vehicle body. Each camera includes an image camera and an event camera, and it should be ensured that the imaging fields of the image camera and the event camera are as close as possible. For example, when designing the camera, it can be ensured that the optical centers and optical axes of the image camera and the event camera are as close as possible. This can be achieved through mechanical structure design and precision assembly. For example, the same lens bracket is used to fix the image camera and the event camera to ensure that their optical axes are parallel and their optical centers coincide. The camera transmits the generated data to the domain control computing device for data processing, that is, the domain control computing device parses the detected code image, implements the parsing of the code image, and obtains the code scanning result information.

[0059] It should be noted that the current event cameras are not mass-produced on vehicles and need to be installed separately. Image cameras are relatively common on vehicles and can be installed separately or directly reuse existing vehicle-mounted cameras, that is, image cameras. For reused image cameras, it is necessary to ensure that it does not affect the operation of other intelligent driving functions of the vehicle. The startup trigger condition of the image camera is to turn on the image camera when the vehicle is turned on. At this time, the image camera continues to collect the first image data for use by downstream functions. In the scenario where the vehicle scans the code, the image camera can collect a frame of first image data when the absolute speed or relative speed of the vehicle indicates that the vehicle is stationary.

[0060] In some embodiments, the identifying the target data by an image recognition unit includes: the image recognition unit receiving the first image data and the event stream data, and identifying the target data based on the first image data and the event stream data.

[0061] Specifically, the image recognition unit receives the first image data and the event stream data, and recognizes the target data according to the first image data and the event stream data to obtain a recognition result.

[0062] In one embodiment, code scanning and recognition can be performed in conjunction with first image data generated by environmental data acquired by an image camera and event stream data generated by environmental data acquired by an event camera to obtain a recognition result. It is understood that in this embodiment, code recognition is performed in conjunction with an image camera and an event camera, leveraging the advantages of event cameras over image cameras in that they are resistant to motion blur, unaffected by illumination, and do not miss moving targets. This ensures that the collected event stream data has superior quality in dynamic vehicle scenarios. Code recognition is performed in conjunction with the first image data, ensuring that the detected code image in the recognition result is of superior quality. This allows for efficient scanning of the code image, enabling effective vehicle code scanning in both static and dynamic scenarios.

[0063] In some embodiments, the target data is identified based on the first image data and the event stream data, including: obtaining the vehicle speed; when the image recognition unit is a camera, code recognition is performed based on the vehicle speed, the first image data and the event stream data.

[0064] The vehicle speed may be obtained by receiving the vehicle speed calculated by the vehicle controller, or by using a sensor on the vehicle.

[0065] Specifically, since the quality of the data collected by the image camera and the event camera will be affected by the vehicle in static and dynamic scenes, code recognition is performed based on the vehicle speed in combination with the first image data collected by the image camera and the event stream data collected by the event camera. This can ensure that the first image data and the event stream data are the best data under the scenario corresponding to the vehicle speed, thereby improving the efficiency of code recognition.

[0066] In some embodiments, the identifying of the target data based on the vehicle speed, the first image data and the event stream data includes: determining the second image data corresponding to the event camera based on the vehicle speed and the event stream data collected by the event camera; identifying the target data based on the vehicle speed, the first image data and the second image data to obtain an identification result.

[0067] The second image data is image data obtained by converting the event stream data collected by the event camera. For example, the second image data may be a grayscale image.

[0068] Specifically, because vehicle speed affects the amount of event stream data collected by the event camera, the event stream data collected by the event camera can be interpolated based on vehicle speed to ensure sufficient event stream data, avoid missing code images due to omissions in event stream data, and improve the quality of the second image data. After ensuring the quality of the second image data, target data can be identified based on vehicle speed, the first image data, and the second image data, which helps improve code recognition efficiency.

[0069] In some embodiments, code recognition is performed based on the vehicle speed, the first image data, and the second image data to obtain a recognition result, including: if the vehicle speed represents that the vehicle is stationary, the target data is recognized based on the first image data and the second image data; if the vehicle speed represents that the vehicle is moving, the target data is recognized based on the second image data to obtain a recognition result.

[0070] Specifically, when the vehicle speed is low or zero, it is determined that the vehicle is stationary, indicating that the vehicle is in a static environment. The target data is then identified based on the first and second image data to obtain a recognition result, which can reduce the missed detection rate of code images and improve code recognition efficiency. When the vehicle speed is high, it is determined that the vehicle speed represents a dynamic environment. In this case, the first image data captured by the image camera will have motion blur. Therefore, the target data is identified based on the second image data, avoiding redundant processing of target data identification based on the first image data and improving recognition efficiency.

[0071] In some embodiments, determining the second image data corresponding to the event camera based on the vehicle speed and the event stream data includes: determining target event stream data based on the vehicle speed and the event stream data; and converting and processing the target event stream data to obtain the second image data.

[0072] Among them, the event camera records the change of light intensity of each pixel in the picture at every moment. If the light intensity exceeds the set threshold, the pixel is assigned a polarity value (usually +1 indicates an increase in brightness, -1 indicates a decrease in brightness). The event camera asynchronously outputs a series of quadruple data, including the pixel coordinates of the event, the timestamp of the event, and the event polarity. The event stream data e m =(x m ,y m , t m , p m ), where (x m ,y m) is the coordinate position of the pixel point, t m is the timestamp information, p m is the polarity value.

[0073] The second image data is event frame image data obtained by converting the event stream data.

[0074] Specifically, because faster vehicle speeds result in shorter time periods, this can lead to data loss or processing delays when the event camera processes events. Therefore, target event stream data can be determined based on vehicle speed and the event stream data collected by the event camera. This can be achieved by dynamically adjusting the time period of the event stream data based on vehicle speed to determine the target event stream data. The target event stream data is then converted into event frame image data to generate second image data. This second image data allows for intuitive observation and analysis of light intensity changes in the scene, improving code recognition performance.

[0075] In some embodiments, the target event stream data is determined based on the vehicle speed and the event stream data, including: if the vehicle speed represents the vehicle movement, and the vehicle speed is less than or equal to a preset speed threshold, the event stream data is determined as the target event stream data; if the vehicle speed represents the vehicle movement, and the vehicle speed is greater than a preset speed threshold, the event stream data is fitted to obtain the target event stream data.

[0076] Specifically, when the vehicle's speed represents vehicle motion, when the vehicle speed is less than or equal to a preset speed threshold, the time period of the event stream data captured by the event camera is determined to be sufficient, and therefore, the event stream data is directly determined as the target event stream data. When the vehicle speed is greater than the preset speed threshold, the time period of the event stream data captured by the event camera is determined to be short, and there may be a problem of missing event stream data leading to missed code image detection. Therefore, the event stream data is fitted, and the fitting processing method can use a time interpolation method to fit event stream data over a longer time period. For example, bilinear interpolation or other interpolation algorithms are used to expand the event stream data on the time axis, thereby obtaining event stream data over a longer time period as the target event stream data. It can be understood that in this embodiment, the target event stream data is determined based on the vehicle speed, so that the time period of the target event stream data is sufficient, thereby ensuring the integrity of the target event stream data.

[0077] In some embodiments, the target event stream data is processed to obtain the second image data, including: determining the target polarity value corresponding to each pixel point based on the polarity value of each pixel point in each frame of the target event stream data; determining the pixel value corresponding to the pixel point based on the target polarity value corresponding to the pixel point to obtain the second image data.

[0078] Specifically, for each pixel in the target event stream data at each timestamp, the polarity value in each frame of the target event stream data is traversed, and the target polarity value of the corresponding pixel is determined based on the polarity value of the pixel in each frame. Then, the corresponding pixel value is determined according to the target polarity value to obtain the second image data. It can be understood that in this embodiment, by analyzing the polarity value of each pixel, the pixel value of each pixel can be determined more accurately. In dynamic scenes, higher quality images can be generated, thereby improving the quality of the second image data.

[0079] In some embodiments, the target polarity value corresponding to each pixel point is determined based on the polarity value of each pixel point in the target event stream data of each frame, including: for each pixel point, counting the polarity value corresponding to each pixel point to obtain the target polarity value corresponding to each pixel point.

[0080] Specifically, for each pixel, statistics are performed based on the polarity value of the corresponding pixel. The statistical method can be to take the average or sum of the polarity values of the same pixel in each frame as the target polarity value of the pixel. It can be understood that this embodiment uses a statistical method to determine the target polarity value of each pixel, which significantly improves the stability and reliability of the polarity value, enhances the image detail and dynamic range, and achieves the goal of improving the visual quality of the image by reducing noise interference and artifacts.

[0081] In some embodiments, the counting of the corresponding polarity values includes: calculating the mean of the polarity values of the corresponding pixel points to obtain the target polarity value.

[0082] Specifically, the polarity values of the corresponding pixel points are averaged and used as the target polarity values of the corresponding pixel points, so that the corresponding pixel values can be subsequently determined based on the target polarity values of the pixel points. It can be understood that this embodiment determines the target polarity value of each pixel point through average calculation, thereby optimizing the calculation efficiency.

[0083] In some embodiments, determining the pixel value corresponding to the pixel point based on the target polarity value corresponding to the pixel point to obtain the second image data includes: setting the pixel value of each pixel point to a first preset pixel value; updating the first preset pixel value of the corresponding pixel point to a second preset pixel value or a third pixel value based on the positive or negative nature of the target polarity value to obtain the second image data.

[0084] Specifically, the pixel values of all pixels can be set to the first preset pixel value, that is, the original pixel values of all pixels are the same, for example, they are all set to RGB values of 128, that is, gray. Then, according to the positive or negative nature of the target polarity value, the first preset pixel value of the corresponding pixel is updated to the second preset pixel value or the third pixel value. For example, when the target polarity value is a positive number, the first preset pixel value of the corresponding pixel is updated to the second preset pixel value, for example, the second pixel value is RGB value 255, that is, white; when the target polarity value is a negative number, the first preset pixel value of the corresponding pixel is updated to the third preset pixel value, for example, the second pixel value is RGB value 0, that is, black. Thereby, the target event stream data is converted into the second image data. As Figure 5 The following figure shows a schematic diagram of two frames of target event stream data and second image data. The left figure shows a schematic diagram of two consecutive frames of target event stream data, and the right figure shows a schematic diagram of the second image data. The second image data is obtained by superimposing the two consecutive frames of target event stream data, namely the start event stream data (ON events) and the end event stream data (OFF events), according to the polarity of the timestamps.

[0085] As can be understood, this embodiment efficiently converts event stream data into image data by setting the initial pixel values of all pixels to a uniform preset value (e.g., gray) and then updating the pixel values based on the positive or negative polarity of the target value. This avoids the complex image reconstruction process and directly utilizes the polarity information in the event stream data to generate the image. This not only improves the efficiency of data conversion but also enhances the contrast and readability of the image, making it suitable for dynamic scenes and resource-constrained environments.

[0086] In a specific embodiment, the target data is identified based on the first image data and the event stream data to obtain an identification result, which can be divided into the following four situations:

[0087] In the first scenario, the target data is identified based on the first image data, and the code image corresponding to the image camera, i.e., the target data, is identified based on the second image data, and the code image corresponding to the event camera, i.e., the target data. In this case, the code image corresponding to the image camera and / or the code image corresponding to the event camera can be directly analyzed and scanned to obtain the scanning result;

[0088] In the second scenario, the target data is identified based on the first image data, and the code image corresponding to the image camera is identified. However, the target data is identified based on the second image data, and the code image corresponding to the event camera is not identified. In this case, the code image corresponding to the image camera can be directly analyzed and scanned to obtain the scanning result.

[0089] In the third scenario, if the target data in the first image data is not used to identify the code image corresponding to the image camera, the target data in the second image data is used to identify the code image corresponding to the event camera. In this case, the code image corresponding to the event camera can be directly parsed and scanned to obtain the scan result.

[0090] In the fourth scenario, if the code image corresponding to the image camera is not identified based on the target data in the first image data, and the code image corresponding to the event camera is not identified based on the target data in the second image data, the steps of invoking the image recognition unit to collect the data to be identified based on the environmental data and identifying the target data in the data to be identified in the above embodiment can be continued until the code image is identified. The identified code image can then be parsed and scanned to obtain a scan result.

[0091] In some embodiments, the data acquisition module includes a radar, and acquiring the environmental data of the vehicle through the data acquisition module includes: acquiring the environmental data through the radar, and generating point cloud data based on the environmental data.

[0092] The radar may be a lidar sensor. The point cloud data is multiple frames of point cloud data generated by the environmental data acquired by the radar, and each frame of point cloud data corresponds to a timestamp.

[0093] The inventors discovered that existing vehicle-mounted camera-based image capture uses passively visible light, making its perception performance significantly affected by changes in ambient light. This can lead to poor image quality in strong or low light conditions (such as at night), resulting in code scanning failures. Radar sensors, which actively emit laser light and are unaffected by ambient lighting conditions, ensure the quality of the collected point cloud data is unaffected by ambient light, improving the efficiency of detecting code images in target data.

[0094] In some embodiments, identifying the target data in the environmental data by an image recognition unit includes: the image recognition unit receiving the point cloud data, and identifying the target data based on the point cloud data.

[0095] Specifically, the image recognition unit receives point cloud data and identifies the target data based on the point cloud data to obtain a recognition result. It is understandable that in this embodiment, code recognition is performed on the point cloud data collected by the radar, taking advantage of the fact that radar sensors are not affected by lighting compared to image cameras. This ensures that the collected point cloud data still has good quality even in poor lighting environments. Code recognition on the point cloud data ensures that the detected code image in the recognition result is of good quality, thereby enabling efficient scanning of the code image, enabling effective vehicle code scanning in poor lighting conditions, and improving the user experience of the code scanning function.

[0096] In some embodiments, identifying the target data based on the point cloud data includes: when the image recognition unit is a radar, converting the point cloud data into a binary image to obtain third image data; and identifying the target data based on the third image data.

[0097] The third image data is an image obtained by processing a binary image corresponding to the point cloud data, such as an image obtained by morphological processing.

[0098] Specifically, the reflection intensity of the point cloud data can be binarized to obtain a corresponding binary image, the binarized image can be morphologically processed to generate third image data, and the target data can be identified based on the third image data to obtain an identification result. It can be understood that this embodiment converts the point cloud data into a binary image and performs morphological processing on the binary image, which simplifies the data processing process. Since the binarization processing can better separate the target code (such as a QR code) and the background, the possibility of misidentification is reduced, and the recognition efficiency and accuracy are improved.

[0099] In some embodiments, converting the point cloud data into a binary image to obtain the third image data includes: performing morphological processing on the binary image to obtain the third image data.

[0100] Specifically, a morphological closing operation can be performed on each binary image. The closing operation includes dilation and erosion to connect the possibly discrete black pixels in the binary image into regions, fill the holes and breaks inside the code image, make the outline of the code image more complete and continuous, and improve the quality of the third image data.

[0101] In some embodiments, converting the point cloud data into a binary image includes: performing denoising on the point cloud data; and performing image processing based on the denoised point cloud data to determine a binary image.

[0102] Among them, the point cloud data at the current moment contains a large number of points in three-dimensional space, and each point has coordinate information (x, y, z) and reflection intensity information.

[0103] Specifically, the point cloud data is denoised to remove outliers in the point cloud data and improve the accuracy and efficiency of subsequent processing. Then, the denoised point cloud data is subjected to image processing, wherein the image processing may be point cloud plane segmentation, calculation of point cloud plane normal vector, projection of point cloud reflection intensity to the image plane, reflection intensity clustering, and reflection intensity binarization processing to obtain a binary image. It can be understood that by denoising the point cloud data before generating the binary image, the quality of the point cloud data and the accuracy of subsequent image processing are improved, misjudgment and artifacts are reduced, and it is beneficial to improve the quality of the binary image.

[0104] In some embodiments, the denoising process on the point cloud data includes: determining outliers of the point cloud data by statistical filtering; and removing the outliers.

[0105] Among them, statistical filtering can adopt filtering based on mean and standard deviation, filtering based on median, etc.

[0106] Specifically, the acquired point cloud data is subjected to denoising to remove outliers and improve the accuracy and efficiency of subsequent processing. Statistical filtering is used to identify outliers in the point cloud data. This involves finding the neighborhood distance of each point in the point cloud data and then determining whether the corresponding point is an outlier based on the neighborhood distance and the corresponding preset distance threshold. This reduces redundant processing of outliers and improves data processing efficiency.

[0107] In some embodiments, determining the neighborhood distance of each point in the point cloud data includes: for each point, searching for at least one neighborhood point corresponding to the point, and calculating the distance between the point and the at least one neighborhood point corresponding to the point to obtain at least one distance; performing Gaussian distribution modeling based on the at least one distance to obtain the neighborhood distance.

[0108] Specifically, for each point in the point cloud data, find its k neighboring points. k is a preset parameter that represents the number of neighboring points considered around each point. Calculate the distance dij from each point to its corresponding k neighboring points, where i represents the number of the point in the point cloud data frame and j represents the number of the neighboring point. Then, based on the k distances, assume that the k distances follow a Gaussian distribution d~N(μ,σ), where μ is the mean distance and σ is the standard deviation of the distance. Through statistical analysis, calculate the neighborhood distance of all points. Then, based on the neighborhood distance and the corresponding preset distance threshold, determine whether the corresponding point is an outlier. The specific process is: determine the mean neighborhood distance of each point. The preset distance threshold can be determined based on the preset confidence level of the Gaussian distribution (such as 95% confidence level). If the mean neighborhood distance is greater than the preset distance threshold, the point is marked as an outlier and removed. The choice of confidence level can be adjusted according to the actual application scenario and noise level.

[0109] In some embodiments, the image processing based on the denoised point cloud data to determine the binary image includes: performing plane segmentation on the denoised point cloud data to obtain a point cloud plane; and performing binarization processing on the point cloud plane to obtain the binary image.

[0110] Specifically, the RANSAC (random sampling consensus) algorithm can be used to perform plane segmentation on the denoised point cloud data. The RANSAC algorithm can extract multiple plane models from the point cloud data, including the ground plane, wall, etc., and screen out the best fitting plane through iterative random sampling and plane fitting, thereby realizing the plane segmentation of the point cloud data and obtaining the point cloud plane. Then, the point cloud plane is binarized so that the obtained binary image can simplify the image while highlighting the structural features of the code image.

[0111] In some embodiments, the binarization processing of the point cloud plane to obtain the binarized image includes: determining the normal vector of the point cloud plane, and determining the reflection intensity of the point cloud plane based on the normal vector; performing clustering processing based on the reflection intensity to obtain cluster centers corresponding to the reflection intensities of different materials on the point cloud plane; and performing binarization processing based on the reflection intensity threshold of the material of the cluster center and the reflection intensity to obtain the binarized image.

[0112] Specifically, after the point cloud plane segmentation is completed, the normal vector of each point cloud plane is calculated according to the plane equation obtained by fitting, where the normal vector is a unit vector perpendicular to the plane, which can indicate the orientation of the plane. By calculating the normal vector, it is helpful to subsequently project the point cloud data onto the image plane, and project the point cloud reflection intensity values of each plane onto the image plane perpendicular to the normal vector. The projection process is to convert the point cloud data in the three-dimensional space into two-dimensional image data, retain the reflection intensity information, that is, determine the reflection intensity, so that the point cloud data can be converted into a form suitable for image processing algorithms. Next, the reflection intensity on each image plane is clustered, and the K-means clustering algorithm can be used. Through clustering, the cluster center of the reflection intensity of different materials on the plane is obtained, thereby determining the threshold for distinguishing different materials. Binarization is performed based on the reflection intensity threshold and reflection intensity of the material at the cluster center to generate a binary image.

[0113] In some embodiments, the reflection intensity threshold of the material based on the cluster center and the reflection intensity are binarized, including: determining that the pixel value of the point corresponding to the reflection intensity greater than the reflection intensity threshold is a fourth preset pixel value, and determining that the pixel value of the point corresponding to the reflection intensity less than or equal to the reflection intensity threshold is a fifth preset pixel value.

[0114] Specifically, pixel values less than or equal to the reflection intensity threshold are set to a fourth preset pixel value. For example, a fourth preset pixel value of 0 indicates low material reflection intensity, likely representing a black area in the code image. Pixel values greater than the reflection intensity threshold are set to a fifth preset pixel value. For example, a fourth preset pixel value of 1 indicates high material reflection intensity, likely representing a white area in the code image. This approach uses the reflection intensity threshold derived through clustering to binarize the reflection intensity on each image plane.

[0115] In some embodiments, the target data is identified based on the first image data and the event stream data to obtain a recognition result, wherein the recognition result is a first recognition result, and the first recognition result includes a first detection identifier indicating whether the code image is recognized and a corresponding first code image; the target data is identified based on the point cloud data to obtain a recognition result, wherein the recognition result is a second recognition result, and the second recognition result includes a second detection identifier indicating whether the code image is recognized and a corresponding second code image.

[0116] The target data is identified according to the first image data and the event stream data to obtain a first identification result, which includes a first detection identifier indicating whether the code image is identified and a corresponding first code image.

[0117] The target data is identified according to the point cloud data to obtain a second identification result, wherein the second identification result includes a second detection identifier indicating whether the code image is identified and a corresponding second code image.

[0118] It should be noted that if the first detection identifier indicates that the code image is not recognized, and the second detection identifier indicates that the code image is not recognized, the steps of identifying the target data based on the first image data and the event stream data, and / or identifying the target data based on the point cloud data are continued until the first code image or the second code image is recognized.

[0119] In one embodiment, Figure 6 The figure shows a flowchart of scanning based on the first recognition result. The target data is identified based on the first image data and the event stream data to obtain the first recognition result. The process of scanning based on the first recognition result is as follows:

[0120] A1. Start process: The process starts and proceeds to the next step of judgment.

[0121] A2. Determine whether the code scanning function is activated: Check whether the code scanning function is activated. Activation can be triggered automatically by a user sending a voice command, clicking the screen, or meeting other preset conditions. If activated, proceed to the next step; otherwise, continue the determination.

[0122] A3. The domain controller continuously obtains vehicle speed information: The domain controller begins to obtain real-time vehicle speed information to provide a basis for subsequent operations.

[0123] A4. Determine whether the current vehicle speed is zero: Based on the acquired vehicle speed information, determine whether the vehicle speed is zero. If it is zero, proceed to the next step; otherwise, proceed directly to the event camera data processing step.

[0124] A5. The image camera (such as a CMOS camera) outputs one frame of RGB image and then stops working: When the vehicle speed is zero, the CMOS camera outputs one frame of RGB image and then stops working, while the event camera continues to work.

[0125] A6. Event camera continuously outputs event stream data: The event camera continuously outputs event stream data, records the changes in light intensity of each pixel, and outputs it in the form of a four-tuple, including pixel coordinates, timestamp, and polarity value.

[0126] A7. Based on the current vehicle speed, the domain controller processes the received event stream data into event frame image data. This process determines the length of a time period based on the current vehicle speed, collects event stream data within that time period, and processes the data into event frame image data. This process includes setting unchanged pixels to 0, recording and averaging polarity changes, and displaying pixel colors based on polarity changes.

[0127] A8. Perform QR code detection and parsing on RGB and event frame images: Use the object detection model to detect QR codes in the RGB and event frame images, and then use common tool algorithms to parse the codes. When the vehicle speed is zero, both images are detected simultaneously; when the vehicle speed is not zero, only the event frame image is detected.

[0128] A9. Output the scan results: Display the scan results on the vehicle screen, or extract and display the QR code image that failed to be parsed.

[0129] A10. Determine whether the code scanning function is deactivated: Check whether the code scanning function is deactivated. If so, end the process; otherwise, return to the step of obtaining vehicle speed information and continue the code scanning operation.

[0130] A11. End process: The process ends.

[0131] In another embodiment, Figure 7 The flowchart of scanning the code based on the second recognition result is shown in FIG. The target data is identified according to the point cloud data to obtain the second recognition result. The process of scanning the code based on the second recognition result is as follows:

[0132] B1, start the process and execute step B2;

[0133] B2. Obtain the laser radar point cloud data frame and execute step B3;

[0134] B3. De-noise the point cloud data and use statistical filtering to remove outliers in the point cloud. Statistical filtering performs statistical analysis on the area of each point and filters out outliers that do not meet the requirements based on the distribution characteristics of the distance from the point to all neighboring points. The main process is as follows:

[0135] 1. Find the k neighboring points of each point;

[0136] 2. Calculate the distance dij from each point to its neighborhood, where i = [1, ..., m] means there are m points in total, and j = [1, ..., k] means each point has k neighbors;

[0137] 3. Model the distance parameters based on the Gaussian distribution d~N(μ,σ), and calculate μ (mean distance) and σ (standard deviation of distance) between all points and their neighbors;

[0138] 4. For each point, calculate the mean distance of its neighborhood. If the mean distance is greater than the specified confidence level of the Gaussian distribution, mark it as an outlier and remove it.

[0139] Go to step B4;

[0140] B4. Use the RANSAC algorithm to perform plane segmentation on the point cloud to segment all planes in the current data frame, including the ground plane, wall plane, etc., and then execute step B5;

[0141] B5. Calculate the normal vectors of each point cloud plane. The point cloud planes have been segmented in the previous step. The normal vectors of each plane can be easily calculated based on the fitted plane equations. Go to step B6.

[0142] B6. Project the point cloud reflection intensity values of each plane onto the image plane perpendicular to the normal vector, and execute step B7;

[0143] B7. Cluster the reflection intensity values on each image plane (a K-means clustering algorithm may be used) to obtain cluster centers of reflection intensities of different materials on the plane, thereby obtaining thresholds for distinguishing different materials, and proceed to step B8.

[0144] B8. Binarize the reflection intensity on each image plane according to a threshold value. Pixel values below the threshold are set to 0, and pixel values above the threshold are set to 1. A value below the threshold indicates low material reflection intensity, possibly representing a black area of the QR code. A value above the threshold indicates high material reflection intensity, possibly representing a white area of the QR code. Execute step B9.

[0145] B9. Perform a morphological closing operation on each binary image to connect the possibly discrete black pixels into regions, and then execute step B10.

[0146] B10. Detect each binary image using a QR code target detection model to obtain QR code location and category information, and then proceed to step B11.

[0147] B11. Parse the detected QR code to obtain the QR code content, and execute step B12;

[0148] B12. End the process.

[0149] In some embodiments, if the first detection mark indicates that a code image is recognized, the corresponding first code image is displayed; and / or if the second detection mark indicates that a code image is recognized, the corresponding second code image is displayed.

[0150] Specifically, if the first detection mark indicates that a code image is recognized, the corresponding first code image is displayed, and / or if the second detection mark indicates that a code image is recognized, the corresponding second code image is displayed. For example, the first code image and / or the second code image can be displayed on the vehicle's onboard screen to analyze and scan the first code image and / or the second code image, thereby achieving effective scanning and improving the user's scanning experience.

[0151] In some embodiments, the method further includes: performing target detection on the first code image to obtain a first coding pattern; and / or performing target detection on the second code image to obtain a second coding pattern.

[0152] Target detection refers to detecting the coded image in the first code image and / or the second code image to extract the coded pattern and obtain the first coded pattern and / or the second coded pattern, thereby facilitating subsequent parsing and scanning processing.

[0153] In some embodiments, performing target detection on the first code image to obtain a first coding pattern and / or performing target detection on the second code image to obtain a second coding pattern includes: performing target detection on the first code image using a trained coding pattern detection model to obtain the first coding pattern; and / or performing target detection on the second code image using a trained coding pattern detection model to obtain the second coding pattern.

[0154] The trained coding pattern detection model can be obtained by training the YOLO model.

[0155] Specifically, the first code image and / or the second code image is input into a trained coding pattern detection model, and the trained coding pattern detection model outputs a first coding pattern and / or a second coding pattern.

[0156] In some embodiments, the method further includes: parsing the first coding pattern to obtain a parsing result; and / or parsing the second coding pattern to obtain a parsing result.

[0157] Specifically, OpenCV or JavaScript libraries such as qrcode-reader and jsQR can be used to perform QR code parsing on the first coding pattern and / or the second coding pattern to obtain the coding pattern content, i.e., the parsing result, thereby completing the vehicle code scanning.

[0158] It is worth mentioning that the analysis results can be displayed on the vehicle's computer screen.

[0159] In some embodiments, before parsing the first coding pattern to obtain a parsing result; and, before parsing the second coding pattern to obtain a parsing result, it also includes: determining the similarity between the first coding pattern and the second coding pattern; when the similarity is greater than a preset similarity threshold, parsing the first coding pattern or the second coding pattern.

[0160] Specifically, OpenCV and the SIFT algorithm can be used to extract feature points from the first and second coding patterns, and the number of matching points can be calculated or the SSIM index can be used to determine similarity. If the similarity is greater than a preset similarity threshold, it indicates that the coding patterns in the first and second coding patterns are similar, and the first or second coding pattern can be parsed, achieving effective code scanning.

[0161] The above-mentioned image recognition method obtains the environmental data of the vehicle through the data acquisition module, and the environmental data includes target data; the target data is recognized by the image recognition unit to obtain a recognition result. Since the target data is recognized by the image recognition unit, the image recognition unit can optimize the data acquisition effect according to the environmental data, so that the quality of the target data is not affected by the ambient light, thereby ensuring the quality of the target data. Therefore, it can improve the detection efficiency of the code image in the target data, improve the vehicle's scanning efficiency, and realize effective vehicle scanning.

[0162] It should be understood that, although the steps in the flowcharts of the above embodiments are shown in sequence as indicated by the arrows, these steps are not necessarily performed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be performed in other orders. Moreover, at least a portion of the steps in the flowcharts of the above embodiments may include multiple steps or multiple stages, and these steps or stages are not necessarily performed at the same time, but can be performed at different times. The execution order of these steps or stages is not necessarily to be performed in sequence, but can be performed in turn or alternately with other steps or at least a portion of steps or stages in other steps.

[0163] Based on the same inventive concept, this application also provides an image recognition system for implementing the image recognition method involved in the above-mentioned embodiment where the electronic device is the execution subject. The implementation solution provided by this device is similar to the implementation solution described in the above-mentioned method. Therefore, the specific limitations in one or more of the image recognition system embodiments provided below can be referred to the limitations of the image recognition method involved in the embodiment where the electronic device is the execution subject, and will not be repeated here.

[0164] In some embodiments, an image recognition system is provided, which includes a data acquisition module and a data recognition unit, and the data acquisition module and the data recognition unit are communicatively connected; the data acquisition module is used to acquire environmental data of the vehicle; and the data recognition unit is used to receive and identify target data in the environmental data.

[0165] In some embodiments, the data acquisition module includes an image camera and an event camera, and the image camera and the event camera are respectively connected to the data identification unit; the image camera is used to acquire the environmental data and generate first image data, and the event camera is used to acquire the environmental data and generate event stream data; the data identification unit is used to receive the first image data and the event stream data, and identify the target data based on the first image data and the event stream data.

[0166] In some embodiments, the data acquisition module includes a radar, which is used to acquire the environmental data and generate point cloud data; the radar is connected to the data identification unit, which is used to receive the point cloud data and identify the target data based on the point cloud data.

[0167] In some embodiments, the target data is graphic coding data, and the graphic coding data includes a one-dimensional code, a two-dimensional code, a barcode, a matrix code, and a trademark code.

[0168] In some embodiments, an electronic device is provided, whose internal structure diagram can be as follows: Figure 8 As shown. The electronic device includes a processor, a memory, an input / output interface, a communication interface, a display unit and an input device. The processor, the memory and the input / output interface are connected via a system bus, and the communication interface, the display unit and the input device are connected to the system bus via the input / output interface. The processor of the electronic device is used to provide computing and control capabilities. The memory of the electronic device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the electronic device is used to exchange information between the processor and an external device. The communication interface of the electronic device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, an image recognition method is implemented.

[0169] Optionally, the electronic device further includes a display unit. The display unit of the electronic device is used to form a visually visible image and can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen. The input device of the electronic device can be a touch layer covering the display screen, a key, trackball, or touchpad provided on the electronic device housing, or an external keyboard, touchpad, or mouse.

[0170] Those skilled in the art will understand that Figure 8The structure shown in the figure is only a block diagram of a part of the structure related to the scheme of the present application, and does not constitute a limitation on the control device to which the scheme of the present application is applied. The specific control device may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.

[0171] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, database or other media used in the embodiments provided in this application may include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory may include random access memory (RAM) or external cache memory, etc. For purposes of illustration and not limitation, RAM may be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The processors involved in the various embodiments provided herein may be general-purpose processors, central processing units (CPUs), graphics processors (GPUs), digital signal processors (DSPs), programmable logic devices (PLDs), data processing logic devices based on quantum computing, and the like, without limitation thereto.

[0172] Correspondingly, an embodiment of the present application also provides an electronic device, which may be a terminal device or a server.

[0173] like Figure 8 As shown, Figure 8Schematic diagram of the structure of an electronic device provided in an embodiment of the present application. The electronic device 1000 includes a processor 1001 having one or more processing cores, a memory 1002 having one or more computer-readable storage media, and a computer program stored in the memory 1002 and executable on the processor. The processor 1001 is electrically connected to the memory 1002. It will be understood by those skilled in the art that the electronic device structure shown in the figure does not constitute a limitation of the electronic device, and may include more or fewer components than shown, or combine certain components, or arrange the components differently.

[0174] The processor 1001 is the control center of the electronic device 1000. It connects the various parts of the entire electronic device 1000 using various interfaces and lines. By running or loading software programs and / or units stored in the memory 1002 and calling data stored in the memory 1002, it executes various functions of the electronic device 1000 and processes data, thereby monitoring the entire electronic device 1000. The processor 1001 can be a central processing unit (CPU), a graphics processing unit (GPU), a network processor (NP), etc., and can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of this application.

[0175] In the embodiment of the present application, the processor 1001 in the electronic device 1000 will load the instructions corresponding to one or more application processes into the memory 1002 according to the following steps, and the processor 1001 will run the application stored in the memory 1002 to implement various functions, such as: obtaining environmental data of the vehicle through the data acquisition module, the environmental data including target data; and identifying the target data through the image recognition unit to obtain a recognition result. The specific implementation of each of the above operations can be found in the previous embodiment and will not be repeated here.

[0176] Alternatively, as Figure 8 As shown, the electronic device 1000 further includes: a touch screen 1003, a radio frequency circuit 1004, an audio circuit 1005, an input unit 1006, and a power supply 1007. Among them, the processor 1001 is electrically connected to the touch screen 1003, the radio frequency circuit 1004, the audio circuit 1005, the input unit 1006, and the power supply 1007 respectively. Those skilled in the art will understand that Figure 9 The electronic device structure shown in the figure does not constitute a limitation to the electronic device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0177] The touch display screen 1003 can be used to display a graphical user interface and receive user actions on the operation instructions generated by the graphical user interface. The touch display screen 1003 can include a display panel and a touch panel. Among them, the display panel can be used to display the information input by the user or the information provided to the user and various graphical user interfaces of the electronic device, and these graphical user interfaces can be composed of graphics, text, icons, videos and any combination thereof. Optionally, the display panel can be configured in the form of a liquid crystal display (LCD), an organic light emitting diode (OLED), etc. The touch panel can be used to collect the user's touch operation on or near it (such as the user uses any suitable object or accessory such as a finger, a stylus on the touch panel or near the touch panel) and generate corresponding operation instructions, and the operation instructions execute corresponding programs. Optionally, the touch panel may include two parts: a touch detection device and a touch controller. Among them, the touch detection device detects the user's touch direction, detects the signal caused by the touch operation, and transmits the signal to the touch controller; the touch controller receives the touch information from the touch detection device, converts it into touch point coordinates, and then sends it to the processor 1001, and can receive commands sent by the processor 1001 and execute them. The touch panel can cover the display panel. When the touch panel detects a touch operation on or near it, it is transmitted to the processor 1001 to determine the type of touch event, and then the processor 1001 provides a corresponding visual output on the display panel according to the type of touch event. In an embodiment of the present application, the touch panel and the display panel can be integrated into the touch display screen 1003 to realize input and output functions. However, in some embodiments, the touch panel and the touch panel can be used as two independent components to realize input and output functions. That is, the touch display screen 1003 can also be used as part of the input unit 1006 to realize the input function.

[0178] The RF circuit 1004 may be used to transmit and receive RF signals, so as to establish wireless communication with a network device or other electronic devices through wireless communication, and to transmit and receive signals with the network device or other electronic devices.

[0179] The audio circuit 1005 can be used to provide an audio interface between the user and the electronic device through a speaker and microphone. The audio circuit 1005 can convert the received audio data into an electrical signal and transmit it to the speaker, which then converts it into a sound signal for output. On the other hand, the microphone converts the collected sound signal into an electrical signal, which is received by the audio circuit 1005 and converted into audio data. The audio data is then output to the processor 1001 for processing, and then sent to another electronic device through the radio frequency circuit 1004, or the audio data is output to the memory 1002 for further processing. The audio circuit 1005 may also include an earphone jack to provide communication between external headphones and the electronic device.

[0180] The input unit 1006 may be configured to receive input digital, character information, or user feature information (such as fingerprint, iris, or facial information), and generate keyboard, mouse, joystick, optical, or trackball signal input related to user settings and function control.

[0181] The power supply 1007 is used to supply power to the various components of the electronic device 1000. Optionally, the power supply 1007 can be logically connected to the processor 1001 via a power management system, thereby enabling the power management system to manage charging, discharging, and power consumption. The power supply 1007 can also include one or more DC or AC power supplies, a recharging system, a power failure detection circuit, a power converter or inverter, a power status indicator, and other arbitrary components.

[0182] although Figure 8 Not shown in the figure, the electronic device 1000 may further include a camera, a sensor, a wireless fidelity module, a Bluetooth module, etc., which will not be described in detail here.

[0183] like Figure 9 As shown, the embodiment of the present application further provides a vehicle 10, which includes the above-mentioned image recognition system, electronic device, or performs the image recognition method described in the above embodiment. The vehicle has all the beneficial effects of the above-mentioned electronic device, etc., and this application will not repeat them here.

[0184] The vehicle may be a fuel vehicle, a plug-in hybrid vehicle or a new energy vehicle, etc., and this application does not make any specific restrictions on this.

[0185] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0186] Those skilled in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be accomplished by instructions, or by controlling related hardware through instructions. The instructions may be stored in a computer-readable storage medium and loaded and executed by a processor.

[0187] To this end, embodiments of the present application provide a computer-readable storage medium storing a plurality of computer programs capable of being loaded by a processor to execute any of the image recognition methods provided in embodiments of the present application. The computer program can execute the following steps of the image recognition method: obtaining environmental data of a vehicle, including target data, through a data acquisition module; and recognizing the target data through an image recognition unit to obtain a recognition result.

[0188] The specific implementation of the above operations can be found in the previous embodiments and will not be repeated here.

[0189] The computer-readable storage medium may include a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0190] Since the computer program stored in the computer-readable storage medium can execute any image recognition method provided in the embodiments of the present application, the beneficial effects that can be achieved by any image recognition method provided in the embodiments of the present application can be achieved. Please refer to the previous embodiments for details and will not be repeated here.

[0191] According to one aspect of the present application, a computer program product or computer program is also provided, the computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of an electronic device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the electronic device to perform the methods provided in various optional implementations of the above embodiments.

[0192] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0193] The above are only preferred embodiments of the present application and do not constitute any form of limitation to the present application. Although the descriptions of each embodiment in the embodiments of the present application have different focuses, for parts that are not described in detail in a certain embodiment, please refer to the relevant embodiments of other embodiments. However, any simple modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present application without departing from the content of the technical solution of the present application are still within the scope of the technical solution of the present application.

Claims

1. An image recognition method, characterized in that: include: Acquire environmental data of the vehicle through a data acquisition module, wherein the environmental data includes target data; The target data is identified by an image recognition unit to obtain a recognition result.

2. The method according to claim 1, characterized in that The data acquisition module includes an image camera and an event camera. The data acquisition module is used to acquire the vehicle's environmental data, including: Acquiring the environmental data through the image camera, and generating first image data according to the environmental data; The environmental data is acquired through the event camera, and event stream data is generated according to the environmental data.

3. The method according to claim 2, characterized in that The identifying the target data by an image recognition unit includes: The image recognition unit receives the first image data and the event stream data, and recognizes the target data based on the first image data and the event stream data.

4. The method according to claim 3, characterized in that Identifying the target data according to the first image data and the event stream data includes: Obtaining the speed of the vehicle; The target data is identified based on the vehicle speed, the first image data, and the event stream data.

5. The method according to claim 4, characterized in that The identifying the target data according to the vehicle speed, the first image data, and the event stream data includes: determining second image data corresponding to the event camera according to the vehicle speed and the event stream data; The target data is recognized based on the vehicle speed, the first image data, and the second image data.

6. The method according to claim 5, characterized in that Identifying the target data according to the vehicle speed, the first image data, and the second image data includes: If the speed of the vehicle indicates that the vehicle is stationary, identifying the target data based on the first image data and the second image data; If the speed of the vehicle represents the movement of the vehicle, target data is identified based on the second image data.

7. The method according to claim 5, characterized in that The determining, according to the vehicle speed and the event stream data, the second image data corresponding to the event camera includes: determining target event stream data according to the vehicle speed and the event stream data; The target event stream data is converted and processed to obtain the second image data.

8. The method according to claim 7, characterized in that The determining target event stream data according to the vehicle speed and the event stream data includes: If the speed of the vehicle represents the movement of the vehicle, and the speed is less than or equal to a preset speed threshold, determining the event stream data as the target event stream data; If the speed of the vehicle represents the movement of the vehicle and the speed is greater than a preset speed threshold, fitting processing is performed on the event stream data to obtain the target event stream data.

9. The method according to claim 7, characterized in that The step of processing the target event stream data to obtain the second image data includes: Determining a target polarity value corresponding to each pixel point according to the polarity value of each pixel point in each frame of the target event stream data; The pixel value corresponding to the pixel point is determined according to the target polarity value corresponding to the pixel point to obtain the second image data.

10. The method according to claim 9, characterized in that Determining the target polarity value corresponding to each pixel point according to the polarity value of each pixel point in the target event stream data of each frame includes: For each of the pixel points, the polarity value corresponding to each of the pixel points is counted to obtain the target polarity value corresponding to each of the pixel points.

11. The method according to claim 10, characterized in that The counting of polarity values corresponding to each pixel point to obtain a target polarity value corresponding to each pixel point includes: The polarity value corresponding to each pixel point is averaged to obtain the target polarity value.

12. The method according to claim 11, characterized in that Determining the pixel value corresponding to the pixel point according to the target polarity value corresponding to the pixel point to obtain the second image data includes: Setting the pixel value of each pixel point to a first preset pixel value; The first preset pixel value of the corresponding pixel point is updated to a second preset pixel value or a third pixel value according to the positive or negative value of the target polarity value to obtain the second image data.

13. The method according to claim 1, wherein The data acquisition module includes a radar, and acquiring the vehicle's environmental data through the data acquisition module includes: The environmental data is acquired through the radar, and point cloud data is generated according to the environmental data.

14. The method according to claim 13, characterized in that Identifying target data in the environmental data by an image recognition unit includes: The image recognition unit receives the point cloud data and recognizes the target data according to the point cloud data.

15. The method according to claim 14, characterized in that The identifying the target data according to the point cloud data includes: Converting the point cloud data into a binary image to obtain third image data; The target data is identified according to the third image data.

16. The method according to claim 15, characterized in that The step of converting the point cloud data into a binary image comprises: Performing denoising processing on the point cloud data; Image processing is performed based on the denoised point cloud data to determine a binary image.

17. The method according to claim 16, characterized in that The denoising process on the point cloud data includes: Determine outliers of the point cloud data by statistical filtering; and remove the outliers.

18. The method according to claim 17, characterized in that The determining of outliers of the point cloud data by statistical filtering includes: Determining a neighborhood distance of each point in the point cloud data; Based on the neighborhood distance of each point and the preset distance threshold corresponding to each point, it is determined whether the corresponding point is an outlier.

19. The method according to claim 18, characterized in that Determining the neighborhood distance of each point in the point cloud data includes: For each point, searching for at least one neighboring point corresponding to each point, and calculating the distance between the point and the at least one neighboring point corresponding to the point to obtain at least one distance; Gaussian distribution modeling is performed based on at least one of the distances to obtain the neighborhood distance.

20. The method according to claim 16, wherein The performing image processing based on the denoised point cloud data to determine a binary image includes: Performing plane segmentation on the denoised point cloud data to obtain a point cloud plane; Binarization is performed on the point cloud plane to obtain the binarized image.

21. The method according to claim 20, characterized in that The binarization processing of the point cloud plane to obtain the binarized image includes: determining a normal vector of the point cloud plane, and determining a reflection intensity of the point cloud plane based on the normal vector; Performing clustering processing based on the reflection intensity to obtain cluster centers corresponding to the reflection intensities of different materials on the point cloud plane; A binarization process is performed based on the reflection intensity threshold of the material of the cluster center and the reflection intensity to obtain the binarized image.

22. The method according to claim 21, characterized in that The binarization processing of the reflection intensity threshold of the material based on the cluster center and the reflection intensity includes: The pixel value of the point corresponding to the reflection intensity greater than the reflection intensity threshold is determined to be a fourth preset pixel value, and the pixel value of the point corresponding to the reflection intensity less than or equal to the reflection intensity threshold is determined to be a fifth preset pixel value.

23. The method according to claim 15, characterized in that The step of converting the point cloud data into a binary image to obtain the third image data includes: Performing morphological processing on the binarized image to obtain the third image data.

24. The method according to claim 4 or 14, characterized in that Identify the target data according to the first image data and the event stream data to obtain a recognition result, wherein the recognition result is a first recognition result, and the first recognition result includes a first detection identifier indicating whether the code image is recognized and a corresponding first code image; The target data is identified according to the point cloud data to obtain an identification result, wherein the identification result is a second identification result, and the second identification result includes a second detection identifier indicating whether the code image is identified and a corresponding second code image.

25. The method according to claim 24, characterized in that Also includes: If the first detection mark indicates that a code image has been recognized, displaying the corresponding first code image; and / or, If the second detection mark indicates that the code image is recognized, the corresponding second code image is displayed.

26. The method according to claim 25, characterized in that Also includes: performing target detection on the first code image to obtain a first coding pattern; and / or, Target detection is performed on the second code image to obtain a second coding pattern.

27. The method according to claim 26, characterized in that performing target detection on the first code image to obtain a first coding pattern; and / or performing target detection on the second code image to obtain a second coding pattern, comprising: Performing target detection on the first code image using a trained code pattern detection model to obtain a first code pattern; and / or, The trained coding pattern detection model is used to perform target detection on the second code image to obtain a second coding pattern.

28. The method according to claim 26, characterized in that Also includes: parsing the first coding pattern to obtain a parsing result; and / or, The second coding pattern is parsed to obtain a parsing result.

29. The method according to claim 28, characterized in that Before parsing the first coding pattern to obtain a parsing result; and parsing the second coding pattern to obtain a parsing result, the method further includes: determining a similarity between the first coding pattern and the second coding pattern; When the similarity is greater than a preset similarity threshold, the first coding pattern or the second coding pattern is parsed.

30. The method according to any one of claims 1 to 29, characterized in that The target data is graphic coding data, which includes one-dimensional code, two-dimensional code, bar code, matrix code and trademark code.

31. An image recognition system, characterized in that include: A data acquisition module and a data identification unit, wherein the data acquisition module and the data identification unit are communicatively connected; The data acquisition module is used to acquire the environmental data of the vehicle; The data identification unit is used to receive and identify target data in the environmental data.

32. The image recognition system according to claim 31, wherein: The data acquisition module includes an image camera and an event camera, and the image camera and the event camera are respectively connected to the data identification unit; The image camera is used to acquire the environmental data and generate first image data, and the event camera is used to acquire the environmental data and generate event stream data; The data identification unit is configured to receive the first image data and the event stream data, and identify the target data according to the first image data and the event stream data.

33. The image recognition system according to claim 31, wherein: The data acquisition module includes a radar, which is used to acquire the environmental data and generate point cloud data; The radar is connected to the data identification unit, and the data identification unit is used to receive the point cloud data and identify the target data according to the point cloud data.

34. The image recognition system according to any one of claims 31 to 33, characterized in that: The target data is graphic coding data, and the graphic coding data includes at least one of a one-dimensional code, a two-dimensional code, a barcode, a matrix code, and a trademark code.

35. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the image recognition method according to any one of claims 1 to 29 is implemented.

36. An electronic device, characterized in that: include: a memory having a computer program stored thereon; A processor, configured to execute the computer program in the memory to implement the image recognition method according to any one of claims 1 to 29.

37. A vehicle, characterized in that: The device comprises the image recognition system according to claims 31 to 34, or the electronic device according to claim 36, or executes the image recognition method according to any one of claims 1 to 29.

Images