Target object inspection method, device, electronic device and storage medium

Abstract

An embodiment of the present application provides a method, device, electronic device and storage medium for inspecting a target object, which can be applied to the transportation field. The method includes: obtaining the position information of a vehicle at the current moment; querying the position information of a target object in a stationary state near the vehicle in a first map according to the position information of the vehicle; determining a first distance between the vehicle and the target object according to the position information of the vehicle and the position information of the target object; when the first distance meets a preset distance threshold, obtaining a first image collected by an on-vehicle camera installed on the vehicle at the current moment, where the first image includes the target object; obtaining an inspection result of the target object according to the operation status information of the target object in the first image. The inspection process of the present application does not require manual inspection, has low cost, and can achieve real-time inspection of the target object, thereby improving the reliability of the inspection.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of intelligent driving technology, and in particular, to a method, device, electronic device, and storage medium for inspecting target objects. Background Art

[0002] In the traffic road network, there are traffic infrastructure facilities that play a crucial role in the safe travel of vehicles and pedestrians, such as traffic light signs, street lights, etc. When these infrastructure facilities fail, they need to be discovered and maintained in a timely manner. Otherwise, it will affect the travel safety of vehicles and pedestrians and even pose potential safety hazards.

[0003] Currently, the manual inspection method is adopted to check each traffic infrastructure facility. This method consumes a large amount of manpower and has a high management cost. Due to the lack of effective monitoring and fault detection means, the operating conditions of traffic infrastructure facilities cannot be monitored in real time. Summary of the Invention

[0004] The embodiments of the present application provide a method, device, electronic device, and storage medium for inspecting target objects, so as to realize real-time inspection of target objects while reducing the inspection cost.

[0005] In a first aspect, the embodiments of the present application provide a method for inspecting target objects, including:

[0006] Obtain the position information of the vehicle at the current moment;

[0007] According to the position information of the vehicle, query the position information of the target objects in a stationary state near the vehicle in the first map;

[0008] According to the position information of the vehicle and the position information of the target objects, determine the first distance between the vehicle and the target objects;

[0009] When the first distance meets a preset distance threshold, obtain the first image collected by the on-vehicle camera installed on the vehicle at the current moment. The first image includes the target objects, and the preset distance threshold is determined based on the height of the target objects;

[0010] Obtain the inspection result of the target objects according to the operation status information of the target objects in the first image.

[0011] In some embodiments, the method of the embodiments of the present application further includes:

[0012] When the inspection result of the target objects is abnormal, send a first message, and the first message includes the position information of the target objects.

[0013] Optionally, the first message further includes the first image.

[0014] Optionally, the first map is a high-precision map, which includes the position information and height information of street lamps, etc.

[0015] In a second aspect, an embodiment of the present application provides an inspection device for a target object, including:

[0016] A first acquisition unit, configured to acquire the position information of the vehicle at the current moment;

[0017] A query unit, configured to query the position information of the target object in a stationary state near the vehicle in the first map according to the position information of the vehicle;

[0018] A determination unit, configured to determine a first distance between the vehicle and the target object according to the position information of the vehicle and the position information of the target object;

[0019] A second acquisition unit, configured to acquire a first image collected by a vehicle-mounted camera on the vehicle at the current moment when the first distance meets a preset distance threshold, where the first image includes the target object, and the preset distance threshold is determined based on the height of the target object;

[0020] An inspection unit, configured to obtain an inspection result of the target object according to the operation status information of the target object in the first image.

[0021] In one embodiment, the inspection unit is specifically configured to determine a position area of the target object in the first image; detect the operation status information of the target object within the position area; and obtain the inspection result of the target object according to the operation status information of the target object.

[0022] In one embodiment, the inspection unit is specifically configured to convert the first image into a grayscale image; perform binarization processing on the grayscale image of the first image; and determine the position area of the target object in the binarized first image according to the position information of the target object.

[0023] In some embodiments, the target object is a street lamp, and the operation status information of the target object includes the spot area of the street lamp. The inspection unit is specifically configured to determine that the inspection result of the street lamp is normal when the spot area of the street lamp is greater than or equal to a preset spot area; and determine that the inspection result of the street lamp is abnormal when the spot area of the street lamp is less than the preset spot area.

[0024] In some embodiments, if the first distance includes the longitudinal distance between the vehicle and the target object (such as a street lamp), the preset distance threshold includes a first longitudinal distance threshold and a second longitudinal distance threshold. The second acquisition unit is specifically configured to acquire the first image collected by the in-vehicle camera at the current moment when it is determined that the longitudinal distance between the vehicle and the target object (such as a street lamp) is greater than the first longitudinal distance threshold and less than the second longitudinal distance threshold.

[0025] In some embodiments, the first distance further includes the lateral distance between the vehicle and the target object (such as a street lamp), and the preset distance threshold further includes a lateral distance threshold. The second acquisition unit is specifically configured to acquire the first image collected by the in-vehicle camera at the current moment when it is determined that the longitudinal distance between the vehicle and the target object (such as a street lamp) is greater than the first longitudinal distance threshold and less than the second longitudinal distance threshold, and the lateral distance between the vehicle and the target object (such as a street lamp) is less than the lateral distance threshold.

[0026] In some embodiments, the above-mentioned determination unit is further configured to obtain the height information of the target object (such as a street lamp) from the first map; and determine the first longitudinal distance threshold and the second longitudinal distance threshold according to the height information of the target object (such as a street lamp).

[0027] In some embodiments, the above-mentioned determination unit is specifically configured to determine the first longitudinal distance threshold according to the height of the target object (such as a street lamp), a first preset value, and the upper viewing angle of the pitch angle of the in-vehicle camera; and determine the second longitudinal distance threshold according to the height of the target object (such as a street lamp), a second preset value, and the upper viewing angle of the pitch angle of the in-vehicle camera.

[0028] Wherein, both the first preset value and the second preset value are greater than 1, and the second preset value is greater than the first preset value.

[0029] In some embodiments, the above-mentioned determination unit is specifically configured to determine the first longitudinal distance threshold according to the product of the height of the target object (such as a street lamp), the first preset value, and the trigonometric function value of the upper viewing angle. For example, the first longitudinal distance threshold is determined according to the following formula:

[0030] L1 = c1 * h1 * cotθ,

[0031] Wherein, L1 is the first longitudinal distance threshold, c1 is the first preset value, h1 is the height of the target object (such as a street lamp), and θ is the upper viewing angle of the pitch angle of the in-vehicle camera.

[0032] In some embodiments, the above-mentioned determining unit is specifically configured to determine the second longitudinal distance threshold according to the product of the height of the target object (such as a street lamp), the second preset value, and the trigonometric function value of the upper viewing angle. For example, the second longitudinal distance threshold is determined according to the following formula:

[0033] L2 = c2 * h1 * cotθ,

[0034] wherein, L2 is the first longitudinal distance threshold, c2 is the second preset value, h1 is the height of the target object (such as a street lamp), and θ is the upper viewing angle of the pitch angle of the vehicle-mounted camera.

[0035] In some embodiments, the above-mentioned determining unit is further configured to, when determining that the longitudinal distance between the vehicle and the target object (such as a street lamp) is greater than the first longitudinal distance threshold and less than the second longitudinal distance threshold, determine the lateral distance threshold according to the longitudinal distance between the vehicle and the target object (such as a street lamp), the third preset value, and the horizontal viewing angle of the vehicle-mounted camera, where the third preset value is a positive number less than 1.

[0036] In some embodiments, the above-mentioned determining unit is specifically configured to determine the lateral distance threshold according to the product of the longitudinal distance between the vehicle and the target object (such as a street lamp), the third preset value, and the trigonometric function value of the horizontal viewing angle. For example, the lateral distance threshold is determined according to the following formula:

[0037] L3 = c3 * d1 * tanα,

[0038] wherein, L3 is the lateral distance threshold, c3 is the third preset value, d1 is the longitudinal distance between the vehicle and the target object (such as a street lamp), and α is half of the horizontal viewing angle of the vehicle-mounted camera.

[0039] In some embodiments, the above-mentioned inspection unit is further configured to send a first message when the inspection result of the target object (such as a street lamp) is abnormal, and the first message includes the position information of the target object (such as a street lamp).

[0040] Optionally, the first message further includes the first image.

[0041] Optionally, the first map is a high-precision map.

[0042] In a third aspect, an embodiment of the present application provides an electronic device, including a processor and a memory;

[0043] The memory is used to store a computer program;

[0044] The processor is configured to execute the computer program to implement the method described in the first aspect above.

[0045] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium. The storage medium includes computer instructions. When the instructions are executed by a computer, the computer is caused to implement the method described in the first aspect.

[0046] In a fifth aspect, an embodiment of the present application provides a computer program product. The program product includes a computer program. The computer program is stored in a readable storage medium. At least one processor of the computer can read the computer program from the readable storage medium. The at least one processor executes the computer program to cause the computer to implement the method described in the first aspect.

[0047] The target object inspection method, device, electronic device, and storage medium provided by the embodiments of the present application obtain the position information of the vehicle at the current moment; query the position information of the target object (such as a street lamp) in a stationary state near the vehicle in a first map according to the position information of the vehicle; determine a first distance between the vehicle and the target object according to the position information of the vehicle and the position information of the target object; when the first distance meets a preset distance threshold, obtain a first image collected by an on-vehicle camera on the vehicle at the current moment, where the first image includes the target object; and obtain an inspection result of the target object according to the operation status information of the target object in the first image. The entire inspection process of the present application is simple, does not require manual inspection, has low costs, and can achieve real-time inspection of the target object, thereby improving the reliability of the inspection. In addition, when it is determined that the first distance between the vehicle and the target object meets the preset distance threshold, the embodiments of the present application obtain the first image collected by the on-vehicle camera at the current moment. That is to say, when it is determined that the first image has the target object, the first image is obtained for processing, thereby reducing the number of images to be processed and improving the efficiency of target object inspection. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] The accompanying drawings here are incorporated into the description and form a part of this description, showing embodiments consistent with the present application and used together with the description to explain the principles of the present application.

[0049] Figure 1 It is a schematic diagram of an Internet of Things system structure related to an embodiment of the present application;

[0050] Figure 2 It is a schematic diagram of an intelligent transportation network related to an embodiment of the present application;

[0051] Figure 3 It is a schematic diagram of an application scenario of an embodiment of the present application;

[0052] Figure 4Schematic flowchart of the method for inspecting target objects of driving risk provided by an embodiment of the present application;

[0053] Figure 5 Schematic diagram of the positioning principle involved in the embodiment of the present application;

[0054] Figure 6A Grayscale image involved in the embodiment of the present application;

[0055] Figure 6B For Figure 6A The grayscale Figure 2 Binary image after value conversion;

[0056] Figure 7 Schematic flowchart of the method for inspecting target objects of driving risk provided by an embodiment of the present application;

[0057] Figure 8A Schematic diagram of the longitudinal distance between the vehicle and the target object involved in the present application;

[0058] Figure 8B Schematic diagram of the lateral distance between the vehicle and the target object involved in the present application;

[0059] Figure 9 Schematic flowchart of the method for inspecting target objects of driving risk provided by another embodiment of the present application;

[0060] Figure 10 Schematic diagram of the inspection system involved in the embodiment of the present application;

[0061] Figure 11 Schematic structural diagram of a target object inspection device provided by the embodiment of the present application;

[0062] Figure 12 Block diagram of the electronic device involved in the embodiment of the present application. Detailed implementation manners

[0063] Next, the technical solutions in the embodiments of the present application will be described in conjunction with the accompanying drawings in the embodiments of the present application.

[0064] To facilitate the understanding of the embodiments of the present application, the following briefly introduces the relevant concepts involved in the embodiments of the present application:

[0065] Vehicle-to-Everything (V2X) provides vehicle information through sensors and in-vehicle terminals installed on vehicles, and enables communication between vehicles (Vehicle to Vehicle, V2V), between vehicles and roads (Vehicle to Infrastructure, V2I), between vehicles and pedestrians (Vehicle to Pedestrian, V2P), and between vehicles and networks (Vehicle to Network, V2N) through various communication technologies.

[0066] Intelligent driving mainly includes three aspects: network navigation, autonomous driving, and assisted driving. The prerequisite for intelligent driving is that the selected vehicle meets the dynamic requirements of driving, the sensors on the vehicle can obtain relevant visual and auditory signals and information, and a corresponding servo system is controlled through cognitive computing.

[0067] Among them, autonomous driving is to complete driving behaviors such as lane keeping, overtaking and lane changing, stopping at red lights and going at green lights, and interacting with light signals and horn signals under the control of an intelligent system.

[0068] Assisted driving means that the driver makes corresponding responses to the actual road conditions under a series of prompts from the intelligent system.

[0069] Traffic Control Unit (TCU for short) constitutes the functional entity of the control subsystem in the intelligent transportation system, coordinates the traffic activities of vehicles, roads, and people based on traffic information, and ensures traffic safety and efficiency. Traffic information includes information about vehicles, pedestrians, roads, facilities, weather, etc., which can be obtained through vehicles, pedestrians, or roadside devices.

[0070] Local Control Unit (LCU for short) is a traffic control device responsible for coordinating traffic activities in a specific area within the management scope of the intelligent transportation system.

[0071] Global Control Unit (GCU for short) is a traffic control device responsible for coordinating traffic activities involving the whole within the management scope of the intelligent transportation system, as well as local traffic control devices.

[0072] Road Side Unit (RSU for short) is a traffic information collection unit or traffic facility control unit deployed near the road. The former provides the collected traffic information to the traffic control device, and the latter executes the control instructions of the traffic control unit for traffic facilities.

[0073] The embodiments of the present application are applied to the field of intelligent driving technology, and are used to perform real-time inspections on the operating status of a target object, so as to reduce the cost of inspecting the target object, and achieve real-time inspection of the target object, thereby improving the inspection efficiency of the target object.

[0074] It should be understood that in the embodiments of the present invention, "B corresponding to A" means that B is associated with A. In one implementation, B can be determined according to A. However, it should also be understood that determining B according to A does not mean determining B only according to A, and B can also be determined according to A and / or other information.

[0075] In the description of the present application, unless otherwise specified, "a plurality of" means two or more than two.

[0076] In addition, in order to facilitate a clear description of the technical solutions of the embodiments of the present application, in the embodiments of the present application, terms such as "first" and "second" are used to distinguish identical or similar items with basically the same functions and effects. Those skilled in the art can understand that terms such as "first" and "second" do not limit the quantity and execution order, and terms such as "first" and "second" do not necessarily limit to be different.

[0077] Figure 1 It is a schematic structural diagram of an Internet of Things system related to the embodiments of the present application, as Figure 1 shown, the Internet of Things system includes: a network-side device 102 and a terminal device. Among them, the terminal device includes in-vehicle terminals 101a, 101b and a user terminal 101c. Here, it is only illustrative and does not specifically limit the Internet of Things system of the embodiments of the present application.

[0078] The in-vehicle terminal may include an on-board computer or an on-board unit (OBU) etc.

[0079] The user terminal (user equipment, UE) 101c can be a wireless terminal device or a wired terminal device. The wireless terminal device can refer to a device with wireless transceiver functions. The user terminal 101c can be a mobile phone, a tablet computer (Pad), a computer with wireless transceiver functions, a virtual reality (VR) user device, an augmented reality (AR) user device, etc., which is not limited here.

[0080] The network-side device 102 can include traffic control devices, base stations, roadside devices, servers, etc. Optionally, the server can be a cloud server.

[0081] The network-side device 102 communicates with the terminal device via a network. For example, the in-vehicle devices 101a and 101b can execute the target inspection method of the embodiments of this application to obtain the inspection results of the target.

[0082] Optionally, the in-vehicle devices 101a and 101b can send the inspection results of the target to the network-side device 102 via the network.

[0083] Optionally, the network-side device 102 can also send the inspection results of the target to the user device 101c via the network.

[0084] Among them, the network can be a 2G, 3G, 4G, 5G communication network or a next-generation communication network.

[0085] Figure 2 This is a schematic diagram of an intelligent transportation network involved in the embodiments of this application. As Figure 2 shown, the intelligent transportation network includes moving vehicles, such as vehicles 11, 12, 19, 20, 21, 22, 23, 24, 25. The intelligent transportation network also includes: traffic control device 14, remote server 15, base station 16, roadside device 17, traffic facilities 18 (such as traffic lights), etc. This is only for illustrative purposes and does not specifically limit the intelligent transportation network.

[0086] In this intelligent transportation network, optionally, wireless communication can be carried out between vehicles and between vehicles and traffic control devices. Wireless communication can also be carried out between traffic control devices, remote servers, roadside devices, and base stations. The remote server or traffic control device can also control traffic facilities, etc. Among them, some vehicles are equipped with on-board computers or OBUs, and some vehicles are equipped with user terminals such as mobile phones. The mobile phones, on-board computers or OBUs in the vehicles can communicate with the network-side device, and here the network-side device can specifically be a traffic control device, a base station, a roadside device, etc.

[0087] A control device can be set on the traffic lights installed at intersections. The control device can control the lighting and extinguishing of different colored lights on the traffic lights. The way the control device controls the lighting and extinguishing of the lights can be: the control device controls according to a preset control mechanism, or the control device receives a control instruction sent by the remote server and controls the lighting and extinguishing of the lights according to the control instruction.

[0088] In this embodiment, the control device can also send the color information of the currently lit traffic signal to the vehicles around the intersection to achieve signal indication. Alternatively, the control device can send the color information of the currently lit traffic signal and the current time to the vehicles around the intersection. Alternatively, the control device can also send the color information of the currently lit traffic signal, the location information of the traffic signal, and the current time to the vehicles around the intersection.

[0089] As Figure 2 shown, the intelligent transportation network can achieve the inspection of the target object. Figure 2 The traffic control devices, base stations, and roadside devices in can be understood as network-side devices, and the vehicle computers or OBUs installed on the vehicles can be called in-vehicle devices, which are used to execute the method of the embodiments of the present application.

[0090] It should be noted that an in-vehicle camera is also installed on the vehicle where the in-vehicle device for executing the method of the embodiments of the present application is located.

[0091] Figure 3 For an application scenario schematic diagram of the embodiments of the present application, as Figure 3 shown, taking the street lamp as an example of the target object, the street lamp is a rod protruding from the ground. When the street lamp fails, it will affect the travel safety of vehicles and pedestrians. Therefore, it is necessary to inspect the street lamp.

[0092] It should be noted that the target objects of the embodiments of the present application include but are not limited to street lamps, and also include traffic infrastructure such as traffic light signs.

[0093] Currently, the manual inspection method is adopted to check each traffic infrastructure. This method consumes a large amount of manpower and has a high management cost. Due to the lack of effective monitoring and fault detection means, the operating conditions of traffic infrastructure cannot be monitored in real time.

[0094] To solve the above technical problems, the embodiments of the present application provide a method for inspecting a target object. By acquiring a first image collected by an intelligent driving vehicle, processing the first image, identifying the operating condition of the target object in the first image, and determining the inspection result of the target object according to the operating condition of the target object, the entire inspection process is simple, without the need for manual inspection, with low cost, and can achieve real-time inspection of the target object, thereby improving the reliability of the inspection. In addition, in the embodiments of the present application, when it is determined that a first distance between the vehicle and the target object satisfies a preset distance threshold, a first image collected by the in-vehicle camera at the current moment is acquired, that is, when it is determined that the target object is present in the first image, the first image is acquired for processing, thereby reducing the number of images to be processed and improving the efficiency of target object inspection.

[0095] The technical solutions of the embodiments of the present application will be described in detail below through some embodiments. These several embodiments may be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0096] Figure 4 The flowchart of the method for inspecting target objects of driving risks provided by an embodiment of the present application. The execution subject of the embodiment of the present application is a device that can determine the inspection result of the target object, such as a target object inspection device, hereinafter referred to as the inspection device for short. In some embodiments, the inspection device is Figure 1 the in-vehicle terminal shown in the figure, such as an in-vehicle computer or an OBU. In some embodiments, the above-mentioned inspection device is a unit with data processing functions in the in-vehicle terminal, such as a processor in the in-vehicle terminal.

[0097] As Figure 4 shown, the method of the embodiment of the present application includes:

[0098] S401. Obtain the position information of the vehicle at the current moment.

[0099] The embodiment of the present application can be applied to the transportation field.

[0100] A positioning system is installed on the vehicle in the embodiment of the present application, and the real-time position information of the vehicle can be obtained from the positioning system.

[0101] In some embodiments, the positioning system installed on the vehicle is a high-precision positioning system.

[0102] In one example, as Figure 5 shown, the positioning system of the vehicle uses sensor inputs such as the Global Navigation Satellite System (GNSS), Inertial Measurement Unit (IMU), and wheel speed (Wheel), and centimeter (cm)-level positioning can be achieved by combining camera or lidar feature matching. As Figure 5 shown, the positioning accuracy based on GNSS positioning is 5 to 10 meters, the positioning accuracy based on dead reckoning positioning of IMU and Wheel is meter-level, and the positioning accuracy based on feature matching positioning of the camera is centimeter-level. Based on the above three positioning methods with different positioning accuracy levels, centimeter-level positioning of the vehicle can be achieved.

[0103] S402. According to the position information of the vehicle, query the position information of the target objects in a stationary state near the vehicle in the first map.

[0104] The target object in the embodiments of the present application is any stationary facility on or beside the road, such as a speed limit sign, a traffic light sign, a traffic indication sign, a street lamp, etc. The embodiments of the present application do not limit the specific type of the target object.

[0105] The above-mentioned target object is a facility that can be captured by the on-vehicle camera of the vehicle in the driving direction of the vehicle. For example, the on-vehicle camera is installed in front of the vehicle, such as on the front windshield of the vehicle. During the driving process of the vehicle, the on-vehicle camera captures the road objects in front of the vehicle and on both sides of the front in real time.

[0106] In one example, the above-mentioned first map is a map on the network side. After the inspection device obtains the position information of the vehicle at the current moment, it accesses the first map through the network and queries the position information of the stationary target objects near the vehicle on the first map.

[0107] In one example, the above-mentioned first map is a map installed on the vehicle, that is, an on-vehicle map. The inspection device can directly query the position information of the stationary target objects near the vehicle on the vehicle map according to the position information of the vehicle at the current moment.

[0108] Optionally, the above-mentioned first map is a high-precision map.

[0109] S403. Determine the first distance between the vehicle and the target object according to the position information of the vehicle and the position information of the target object.

[0110] In one example, the first distance between the vehicle and the target object may include the Euclidean distance between the vehicle and the target object.

[0111] In one example, the first distance between the vehicle and the target object may include the longitudinal distance between the vehicle and the target object.

[0112] In one example, the first distance between the vehicle and the target object may include the lateral distance between the vehicle and the target object.

[0113] In one example, the first distance between the vehicle and the target object may include the longitudinal distance and the lateral distance between the vehicle and the target object.

[0114] The embodiments of the present application do not limit the specific type of the above-mentioned first distance.

[0115] S404. When the first distance meets the preset distance threshold, acquire the first image collected by the on-vehicle camera installed on the vehicle at the current moment, and the first image includes the target object.

[0116] Since the in-vehicle camera collects the road map of the vehicle in real time, for example, collects a road map every 3 seconds, the data volume is large, and there is a large amount of data unrelated to the inspection of the target object. In order to reduce the data processing volume, when the first distance between the vehicle and the target object at the current moment meets the preset distance threshold, the present application acquires the first image collected by the vehicle camera at the current moment, where the first image includes the target object.

[0117] The present application does not limit the preset distance threshold, as long as when the first distance between the vehicle and the target object meets the preset distance threshold, the first image collected by the in-vehicle camera includes the target object.

[0118] Optionally, the preset distance threshold is determined based on the height of the target object.

[0119] S405. Obtain the inspection result of the target object according to the operation status information of the target object in the first image.

[0120] After the inspection device acquires the first image, it identifies the state of the target object in the first image to obtain the operation status information of the target object at the current moment. According to the operation status information of the target object in the first image, the inspection result of the target object is obtained.

[0121] Optionally, the inspection result of the target object includes two types: normal and abnormal.

[0122] In some embodiments, the above S405 obtaining the inspection result of the target object according to the operation status information of the target object in the first image includes:

[0123] S405-A1. Determine the position area of the target object in the first image;

[0124] S405-A2. Detect the operation status information of the target object within the position area;

[0125] S405-A3. Obtain the inspection result of the target object according to the operation status information of the target object.

[0126] In this embodiment, after acquiring the first image, in order to further reduce the data volume to be processed, through geometric transformation, the position area of the target object in the first image can be determined according to the position information of the target object. Detect the operation status information of the target object within this position area, rather than detecting the operation status information of the target object in the entire first image, thereby reducing the data processing volume and improving the determination speed of the operation status information of the target object.

[0127] In order to further improve the detection efficiency of the operation status of the target object, the implementation manner of the above S405 includes but is not limited to the following several ways:

[0128] Method 1: Convert the first image into a grayscale image; perform binarization on the grayscale image of the first image; determine the position area of the target object in the binarized first image according to the position information of the target object; detect the operation status information of the target object within this position area, and determine the inspection result of the target object according to the operation status information of the target object.

[0129] For example, taking the target object as a street lamp, first convert the first image collected by the vehicle-mounted camera into Figure 6A the grayscale image as shown, and then set the threshold of the grayscale value to binarize the Figure 6A grayscale Figure 2 value into the black-and-white binarized image as shown. According to the position information of the target object, determine the position area of the target object in the Figure 6B binarized image as shown, such as the Figure 6B box shown. Detect the operation status information of the target object within this position area to obtain the inspection result of the target object. Figure 6B

[0130] Since a single pixel of a color image is (R, G, B) and is converted into a grayscale image (L), the matrix can be greatly simplified and the operation speed can be improved.

[0131] Optionally, the average value method can be used to obtain a grayscale value by averaging the three-component brightness in the color image.

[0132] Method 2: Determine the position area of the target object in the first image according to the position information of the target object; convert the position area into a grayscale image; perform binarization on the position area converted into a grayscale image.

[0133] First, determine the position area of the target object in the first image according to the position information of the target object, convert this position area from a color image into a grayscale image, and then set the threshold of the grayscale value to binarize the Figure 2 grayscale value into a black-and-white binarized image. Detect the operation status information of the target object in this binarized image.

[0134] The specific type of the target object is not limited in the embodiments of the present application.

[0135] In one example, the above target object is a street lamp.

[0136] When the target object is a street lamp, the operation status information of the target object includes the light spot area of the street lamp. At this time, the above S405-A3 includes: when the light spot area of the street lamp is greater than or equal to the preset light spot area, determining that the inspection result of the target object is normal; when the light spot area of the street lamp is less than the preset light spot area, determining that the inspection result of the target object is abnormal.

[0137] For example, as Figure 6B ​As shown, the white spot in the detection frame is the light spot area S of the street lamp. When the light spot area S > Sth, it is determined that the street lamp is lit and working properly; otherwise, it is determined that the street lamp is working abnormally.

[0138] The method of the embodiment of the present application includes: obtaining the position information of the vehicle at the current moment; according to the position information of the vehicle, querying the position information of the target objects in a stationary state near the vehicle in the first map; determining the first distance between the vehicle and the target objects according to the position information of the vehicle and the position information of the target objects; when the first distance meets the preset distance threshold, obtaining the first image collected by the on-vehicle camera installed on the vehicle at the current moment, where the first image includes the target objects; and obtaining the inspection result of the target objects according to the operation status information of the target objects in the first image. The entire inspection process of the present application is simple, without the need for manual inspection, with low cost, and can achieve real-time inspection of the target objects, thereby improving the reliability of the inspection. In addition, in the embodiment of the present application, when it is determined that the first distance between the vehicle and the target objects meets the preset distance threshold, the first image collected by the on-vehicle camera at the current moment is obtained. That is to say, when it is determined that the first image has the target objects, the first image is obtained for processing, thereby reducing the number of images to be processed and improving the inspection efficiency of the target objects.

[0139] Figure 7 is a schematic flowchart of the method for inspecting target objects of driving risks provided by an embodiment of the present application. As Figure 7 shown, the method of the embodiment of the present application includes:

[0140] S701. Obtain the position information of the vehicle at the current moment.

[0141] S702. According to the position information of the vehicle, query the position information of the target objects in a stationary state near the vehicle in the first map.

[0142] The implementation processes of the above S701 and S702 are the same as those of the above S401 and S402. Refer to the descriptions of the above S401 and S402, and details will not be repeated here.

[0143] S703. Determine the first distance between the vehicle and the target objects (such as street lamps) according to the position information of the vehicle and the position information of the target objects (such as street lamps).

[0144] S704. When the first distance meets the preset distance threshold, obtain the first image collected by the on-vehicle camera installed on the vehicle at the current moment, where the first image includes the target objects.

[0145] The first distance between the vehicle and the target objects (such as street lamps) in the embodiment of the present application includes the following situations:

[0146] Case 1: The first distance between the vehicle and the target object (such as a street lamp) includes the longitudinal distance between the vehicle and the target object (such as a street lamp). Correspondingly, the preset distance threshold includes a first longitudinal distance threshold and a second longitudinal distance threshold.

[0147] In Case 1, the above S704 includes: when it is determined that the longitudinal distance between the vehicle and the target object (such as a street lamp) is greater than the first longitudinal distance threshold and less than the second longitudinal distance threshold, it can be determined that the in-vehicle camera can capture the target object (such as a street lamp). At this time, the first image collected by the in-vehicle camera at the current moment is obtained, and the running state of the target object (such as a street lamp) at the current moment is judged according to the first image. Compared with processing all the images collected by the in-vehicle camera to judge the running state of the target object at the current moment, this embodiment can greatly reduce the data processing volume.

[0148] In addition, when it is determined that the longitudinal distance between the vehicle and the target object (such as a street lamp) is less than the first longitudinal distance threshold, or the longitudinal distance between the vehicle and the target object (such as a street lamp) is greater than the second longitudinal distance threshold, it can be determined that the in-vehicle camera cannot capture the target object. At this time, the first image collected by the in-vehicle camera is not obtained, thereby preventing the processing of invalid data to save computing resources.

[0149] In a possible implementation manner, the above first longitudinal distance threshold and / or second longitudinal distance threshold may be preset values.

[0150] In a possible implementation manner, the above first longitudinal distance threshold and / or second longitudinal distance threshold are inferred from historical data. For example, the images collected by the vehicle at historical moments are processed to judge the longitudinal distance range between the vehicle and the target object (such as a street lamp) when the in-vehicle camera can capture the target object. Then, according to the longitudinal distance range between the vehicle and the target object (such as a street lamp) when the in-vehicle camera can capture the target object, the first longitudinal distance threshold and the second longitudinal distance threshold are determined. For example, the first longitudinal distance threshold is the minimum value in the above longitudinal distance range, and the second longitudinal distance threshold is the maximum value in the above longitudinal distance range.

[0151] In a possible implementation manner, the method of the embodiment of the present application further includes:

[0152] Step A1: Obtain the height information of the target object (such as a street lamp) from the first map;

[0153] Step A2: Determine the first longitudinal distance threshold and the second longitudinal distance threshold according to the height information of the target object (such as a street lamp).

[0154] In some embodiments, the above step A2 includes: determining a first longitudinal distance threshold according to the height of the target object (such as a street lamp), a first preset value, and the upper viewing angle of the pitching angle of the vehicle-mounted camera; determining a second longitudinal distance threshold according to the height of the target object, a second preset value, and the upper viewing angle of the pitching angle of the vehicle-mounted camera.

[0155] Wherein, both the first preset value and the second preset value are greater than 1, and the second preset value is greater than the first preset value.

[0156] It should be noted that the embodiments of the present application do not limit the specific values of the first preset value and the second preset value, as long as both the first preset value and the second preset value are greater than 1, and the second preset value is greater than the first preset value.

[0157] Optionally, the first preset value and the second preset value are greater than 1 and less than 2.

[0158] In one example, as Figure 8A shown, the vehicle-mounted camera is installed on the front windshield of the vehicle. The field of view angle of the vehicle-mounted camera is determined by the optical parameters of the vehicle-mounted camera. For example, the upper viewing angle (i.e., the vertical upper viewing angle) of the pitching angle of the vehicle-mounted camera is set as θ. As Figure 8A shown, the first longitudinal distance threshold and the second longitudinal distance threshold can be determined according to the height of the target object (such as a street lamp) and the upper viewing angle of the pitching angle of the vehicle-mounted camera.

[0159] In a possible implementation manner, the first longitudinal distance threshold can be determined according to the product of the height of the target object, the first preset value, and the trigonometric function value of the upper viewing angle.

[0160] For example, the first longitudinal distance threshold is determined according to the following formula (1):

[0161] L1 = c1 * h1 * cotθ (1)

[0162] Wherein, L1 is the first longitudinal distance threshold, c1 is the first preset value, h1 is the height of the target object (such as a street lamp), and θ is the upper viewing angle of the pitching angle of the vehicle-mounted camera.

[0163] It should be noted that the above formula (1) is a way to determine the first longitudinal distance threshold according to the height of the target object (such as a street lamp), the first preset value, and the upper viewing angle of the pitching angle of the vehicle-mounted camera. Any deformation of the above formula (1) also belongs to the protection scope of the embodiments of the present application. For example, when performing equivalent deformation on the above formula (1), or multiplying, dividing, adding, or subtracting a certain value to the above formula (1), it also belongs to the protection scope of the embodiments of the present application.

[0164] In a possible implementation, the second longitudinal distance threshold can be determined based on the product of the height of the target object (such as a street lamp), the second preset value, and the trigonometric function value of the upper viewing angle.

[0165] For example, according to the following formula (2), the second longitudinal distance threshold is determined:

[0166] L2 = c2 * h1 * cotθ (2)

[0167] Where, L2 is the first longitudinal distance threshold, c2 is the second preset value, h1 is the height of the target object (such as a street lamp), and θ is the upper viewing angle of the pitch angle of the vehicle-mounted camera.

[0168] It should be noted that the above formula (2) is a way to determine the second longitudinal distance threshold based on the height of the target object (such as a street lamp), the second preset value, and the upper viewing angle of the pitch angle of the vehicle-mounted camera. Any deformation of the above formula (2) also belongs to the protection scope of the embodiments of the present application. For example, when performing an equivalent deformation on the above formula (2), or multiplying, dividing, adding, or subtracting a certain value to the above formula (2), it also belongs to the protection scope of the embodiments of the present application.

[0169] Optionally, to ensure that the target object (such as a street lamp) can be completely included in the first image, the height of the first image should be 1.3 to 1.5 times the height of the target object (such as a street lamp).

[0170] Optionally, the above first preset value = 1.3.

[0171] Optionally, the above second preset value = 1.5.

[0172] Exemplarily, taking the target object (such as a street lamp) as a street lamp, the height h1 of the street lamp is obtained according to the high-precision map, and the maximum vertical height of the image is taken as 1.3 - 1.5 times the height of the street lamp, which can ensure that the position of the vehicle-mounted camera can be effectively distinguished from the vehicle lights. Then the longitudinal distance d1 between the vehicle and the street lamp should be within 1.3h1 * cotθ < d1 < 1.5h1 * cotθ.

[0173] Case 2, if the lane is too wide and the vehicle is close to the center of the road, it is possible that the field of view of the vehicle-mounted camera cannot cover the target, etc. In addition, there are usually large distortions at the edges of the field of view of the vehicle-mounted camera, which affects image recognition. Therefore, when obtaining the first image, it is also necessary to judge the lateral distance between the vehicle and the target object (such as a street lamp). That is to say, in Case 2, the first distance between the vehicle and the target object (such as a street lamp) includes not only the longitudinal distance between the vehicle and the target object (such as a street lamp) but also the lateral distance between the vehicle and the target object (such as a street lamp). Correspondingly, the preset distance threshold includes not only the first longitudinal distance threshold and the second longitudinal distance threshold but also the lateral distance threshold.

[0174] In Case 2, the above S704 includes: when it is determined that the longitudinal distance between the vehicle and the target object (such as a street lamp) is greater than the first longitudinal distance threshold and less than the second longitudinal distance threshold, and the lateral distance between the vehicle and the target object (such as a street lamp) is less than the lateral distance threshold, obtaining the first image collected by the in-vehicle camera at the current moment.

[0175] That is, in Case 2, when it is judged that the longitudinal distance between the vehicle and the target object (such as a street lamp) is greater than the first longitudinal distance threshold and less than the second longitudinal distance threshold, and the lateral distance between the vehicle and the target object (such as a street lamp) is less than the lateral distance threshold, it can be determined that the target object (such as a street lamp) is within the field of view of the in-vehicle camera, and the in-vehicle camera can capture the target object (such as a street lamp). At this time, obtaining the first image collected by the in-vehicle camera can ensure that the first image includes the target object (such as a street lamp), preventing the ineffective processing of images that do not include the target object (such as a street lamp) and causing waste of computing resources.

[0176] In addition, when it is determined that the longitudinal distance between the vehicle and the target object (such as a street lamp) is less than the first longitudinal distance threshold, or the longitudinal distance between the vehicle and the target object (such as a street lamp) is greater than the second longitudinal distance threshold, it can be judged that the in-vehicle camera cannot capture the target object. Alternatively, when it is determined that the longitudinal distance between the vehicle and the target object (such as a street lamp) is greater than the first longitudinal distance threshold and less than the second longitudinal distance threshold, and the lateral distance between the vehicle and the target object (such as a street lamp) is greater than or equal to the lateral distance threshold, it can be judged that the in-vehicle camera cannot capture the target object (such as a street lamp). When it is determined that the in-vehicle camera cannot capture the target object (such as a street lamp), the first image collected by the in-vehicle camera is not obtained, thereby preventing the processing of invalid data to save computing resources.

[0177] In a possible implementation manner, the above lateral distance threshold may be a preset value.

[0178] In a possible implementation manner, the above lateral distance threshold is inferred from historical data. For example, the images collected by the vehicle at historical moments are processed to judge the lateral distance between the vehicle and the target object (such as a street lamp) when the in-vehicle camera can capture the target object. According to the lateral distance between the vehicle and the target object (such as a street lamp) when the in-vehicle camera can capture the target object, the lateral distance threshold between the vehicle and the target object (such as a street lamp) is determined. For example, the lateral distance between the vehicle and the target object (such as a street lamp) is determined as the lateral distance threshold between the vehicle and the target object (such as a street lamp).

[0179] In a possible implementation, the method of the embodiments of the present application further includes: when it is determined that the longitudinal distance between the vehicle and the target object (such as a street lamp) is greater than a first longitudinal distance threshold and less than a second longitudinal distance threshold, determining a lateral distance threshold according to the longitudinal distance between the vehicle and the target object (such as a street lamp), a third preset value, and the horizontal viewing angle of the vehicle-mounted camera, where the third preset value is a positive number less than 1.

[0180] When it is determined that the longitudinal distance between the vehicle and the target object (such as a street lamp) is less than the first longitudinal distance threshold or greater than the second longitudinal distance threshold, it can be determined that the target object is not within the field of view of the vehicle-mounted camera. At this time, the image collected by the vehicle-mounted camera does not include the target object (such as a street lamp), so there is no need to determine the lateral distance threshold. When it is determined that the longitudinal distance between the vehicle and the target object (such as a street lamp) is greater than the first longitudinal distance threshold and less than the second longitudinal distance threshold, when the lateral distance between the vehicle and the target object (such as a street lamp) meets the lateral distance threshold, the vehicle-mounted camera can collect the target object (such as a street lamp). Therefore, when it is determined that the longitudinal distance between the vehicle and the target object (such as a street lamp) is less than the first longitudinal distance threshold or greater than the second longitudinal distance threshold, it is determined

[0181] It should be noted that the embodiments of the present application do not limit the specific value of the third preset value, as long as the third preset value is a positive number less than 1.

[0182] Optionally, the third preset value is greater than 0.5 and less than 1.

[0183] In one example, as Figure 8B shown, the vehicle-mounted camera is installed on the front windshield of the vehicle. The viewing angle of the vehicle-mounted camera is determined by the optical parameters of the vehicle-mounted camera. For example, half of the horizontal viewing angle of the vehicle-mounted camera (i.e., the horizontal half-viewing angle) is set as α.

[0184] As Figure 8B shown, exemplarily, the lateral distance threshold can be determined according to the distance between two points of the vehicle and the target object (such as a street lamp) and half of the horizontal viewing angle of the vehicle-mounted camera.

[0185] As Figure 8B shown, exemplarily, the lateral distance threshold can be determined according to the longitudinal distance between the vehicle and the target object (such as a street lamp) and half of the horizontal viewing angle of the vehicle-mounted camera.

[0186] In a possible implementation, the lateral distance threshold can be determined according to the product of the longitudinal distance between the vehicle and the target object (such as a street lamp), the third preset value, and the trigonometric function value of the horizontal viewing angle.

[0187] For example, according to the following formula (3), the lateral distance threshold is determined:

[0188] L3 = c3 * d1 * tanα (3)

[0189] Wherein, L3 is the horizontal distance threshold, c3 is the third preset value, d1 is the longitudinal distance between the vehicle and the target object, and α is half of the horizontal viewing angle of the vehicle-mounted camera.

[0190] Optionally, the third preset value c3 = 0.8.

[0191] It should be noted that the above formula (3) is a way to determine the horizontal distance threshold based on the longitudinal distance between the vehicle and the target object (such as a street lamp), the third preset value, and the horizontal viewing angle of the vehicle-mounted camera. Any deformation of the above formula (3) also belongs to the protection scope of the embodiments of the present application. For example, when performing an equivalent deformation on the above formula (3), or multiplying, dividing, adding, or subtracting a certain value from the above formula (3), it also belongs to the protection scope of the embodiments of the present application.

[0192] According to the above method, when it is determined that the longitudinal distance between the vehicle and the target object (such as a street lamp) is greater than the first longitudinal distance threshold and less than the second longitudinal distance threshold, and the horizontal distance between the vehicle and the target object (such as a street lamp) is less than the horizontal distance threshold, it is determined that the first image collected by the vehicle-mounted camera includes the target object. Then, the first image collected by the vehicle-mounted camera at the current moment is obtained, and the following S705 and S706 are then executed.

[0193] S705. Obtain the inspection result of the target object according to the operation status information of the target object (such as a street lamp) in the first image.

[0194] For example, convert the first image into a grayscale image, perform binarization processing on the grayscale image of the first image, perform coordinate transformation, determine the position area of the target object (such as a street lamp) in the binarized first image according to the position information of the target object (such as a street lamp), detect the operation status information of the target object (such as a street lamp) in this position area, and determine the inspection result of the target object (such as a street lamp) according to the operation status information of the target object (such as a street lamp).

[0195] The specific implementation process of the above S705 refers to the description of the above S405 and will not be elaborated here.

[0196] S706. When the inspection result of the target object (such as a street lamp) is abnormal, send the first information.

[0197] Optionally, the first information includes the position information of the target object (such as a street lamp).

[0198] Optionally, the first information further includes the first image.

[0199] In a possible implementation, when the inspection result of the target object (such as a street lamp) is abnormal, the vehicle sends the first information to the cloud, and the cloud triggers a maintenance work order and sends the maintenance work order to the corresponding maintenance unit, so that the maintenance unit performs maintenance on the target object according to the maintenance work order.

[0200] Further, in a specific embodiment, taking the target object as a street lamp as an example, as Figure 9 shown, the method of the embodiment of the present application includes:

[0201] A1. Read the position information of the vehicle, for example, obtain the position information of the vehicle from the positioning module of the vehicle.

[0202] A2. According to the position information of the vehicle, load the first map, and read the position information, height information, etc. of the street lamps near the vehicle in the first map. Optionally, the first map is a high-precision map. The high-precision map includes information such as the position information of the street lamps and the height of the street lamps.

[0203] A3. According to the position information of the vehicle, as well as the position information and height information of the street lamp, determine whether to enter the photographing position, that is, according to the position information of the vehicle, as well as the position information and height information of the street lamp, determine whether the image collected by the on-vehicle camera includes the target object.

[0204] Specifically, according to the product of the height of the street lamp, the first preset value, and the trigonometric function value of the upper view angle of the pitch angle of the on-vehicle camera, determine the first longitudinal distance threshold. For example, the first longitudinal distance threshold L1 = c1 * h1 * cotθ. According to the product of the height of the target object, the second preset value, and the trigonometric function value of the upper view angle of the pitch angle of the on-vehicle camera, determine the second longitudinal distance threshold. For example, the second longitudinal distance threshold L2 = c2 * h1 * cotθ. According to the position information of the vehicle and the position information of the street lamp, determine the longitudinal distance d1 between the vehicle and the street lamp. If c1 * h1 * cotθ < longitudinal distance d1 < c2 * h1 * cotθ, it is determined that the vehicle is longitudinally within the shooting range of the on-vehicle camera.

[0205] Next, judge the lateral position of the vehicle from the street lamp. Specifically, according to the product of the longitudinal distance between the vehicle and the target object, the third preset value, and the trigonometric function value of the horizontal view angle, determine the lateral distance threshold. For example, the lateral distance threshold L3 = c3 * d1 * tanα. According to the position information of the vehicle and the position information of the street lamp, determine the lateral distance d2 between the vehicle and the street lamp. If c3 * d1 * tanα < lateral distance d2, it is determined that the vehicle is laterally within the shooting range of the on-vehicle camera.

[0206] When it is determined that the vehicle is longitudinally and laterally within the shooting range of the on-vehicle camera, that is, when it is determined that the first image collected by the on-vehicle camera includes the street lamp, step A4 is executed.

[0207] A4. Read the first image collected by the vehicle-mounted camera, where the first image includes street lights.

[0208] A5. Perform grayscale conversion on the first image.

[0209] A6. Perform binarization processing on the grayscale-converted first image.

[0210] A7. Perform coordinate conversion according to the position information of the street lights.

[0211] A8. According to the position information of the street lights after coordinate system conversion, determine the position area of the street lights in the first image, and determine whether there is a light spot in this position area. If it is determined that there is no light spot in this position area, execute A10; if it is determined that there is a light spot in this position area, execute A9.

[0212] A9. Determine whether the area of the light spot in the position area is greater than the threshold; if it is determined that the light spot in the position area is greater than or equal to the threshold, determine that the inspection result of the street light is normal, return to execute A1, and continue to inspect the next street light. If it is determined that the light spot in the position area is less than the threshold, determine that the inspection result of the street light is abnormal. At this time, execute A10.

[0213] A10. Send the first information, where the first information includes the position information of the street light and the first image.

[0214] A11. Trigger a maintenance work order.

[0215] In this embodiment, when a street light fails, the faulty street light can be quickly inspected through the above method. The entire inspection process is simple, without the need for manual inspection, with low cost, and real-time inspection of street lights can be achieved, thereby improving the reliability of the inspection.

[0216] In one embodiment, the system structure diagram of the embodiment of the present application is as Figure 10 shown, including a positioning module, a processing module, and a communication module.

[0217] Optionally, the positioning module includes a camera, an IMU, wheel speed, and GNSS.

[0218] Optionally, the processing module may include an ADCU (Auxiliary and Autopilot Control Unit).

[0219] Optionally, the communication module may include a 4G / 5G communication module.

[0220] Among them, the camera perception positioning combined with IMU, wheel rotation, and GNSS positioning can achieve centimeter-level accurate positioning.

[0221] Using the above Figure 10When the shown system performs street lamp inspection, the processing module executes the steps A1 to A10 above. This processing module can be understood as the above-mentioned inspection device or a part of the inspection device. The inspection process includes: the processing module obtains the position information of the vehicle at the current moment from the positioning module, and the map engine in the processing module loads a high-precision map to obtain the position information, height information, etc. of the street lamps.

[0222] The processing module determines whether it enters the photographing position according to the position information and height information of the street lamp and the position information of the vehicle. If it is determined to enter the photographing position, it reads the first image collected by the in-vehicle camera at the current moment, processes the first image to obtain the operation status information of the street lamp, and determines the inspection result of the street lamp according to the operation status information of the street lamp. When the inspection result of the street lamp is abnormal, it sends a first message through the communication module. For example, it sends the first message to the cloud through the communication module. The first message includes the position information of the street lamp and the first image, etc.

[0223] The method of the embodiment of the present application includes: obtaining the position information of the vehicle at the current moment; querying the position information of the target object in the first map according to the position information of the vehicle; when it is determined that the longitudinal distance between the vehicle and the target object is greater than the first longitudinal distance threshold and less than the second longitudinal distance threshold, and the lateral distance between the vehicle and the target object is less than the lateral distance threshold, obtaining the first image collected by the in-vehicle camera at the current moment; obtaining the inspection result of the target object according to the operation status information of the target object in the first image, and sending a first message when the inspection result is abnormal. The first message includes the position information of the target object and the first image. In this embodiment, when it is determined that the longitudinal distance between the vehicle and the target object is greater than the first longitudinal distance threshold and less than the second longitudinal distance threshold, and the lateral distance between the vehicle and the target object is less than the lateral distance threshold, the first image collected by the in-vehicle camera at the current moment is obtained, preventing invalid processing of images that do not include the target object, thereby saving computing resources and improving the efficiency and accuracy of target object inspection.

[0224] The preferred embodiments of the present application have been described in detail above with reference to the accompanying drawings. However, the present application is not limited to the specific details in the above embodiments. Within the scope of the technical concept of the present application, various simple modifications can be made to the technical solutions of the present application, and these simple modifications all belong to the protection scope of the present application. For example, in the various specific technical features described in the above specific embodiments, they can be combined in any suitable manner without contradiction. To avoid unnecessary repetition, the present application does not explain various possible combination methods separately. Also, for example, any combination can be made between various different embodiments of the present application as long as it does not violate the idea of the present application, and it should also be regarded as the content disclosed by the present application.

[0225] It should also be understood that in various method embodiments of the present application, the magnitudes of the sequence numbers of the above processes do not indicate the sequence of execution. The execution sequence of each process should be determined according to its function and internal logic, and should not constitute any limitation to the implementation process of the embodiments of the present application.

[0226] Figure 11 FIG. 4 is a schematic structural diagram of an object inspection device provided by an embodiment of the present application. The inspection device may be an electronic device or a component of an electronic device (e.g., an integrated circuit, a chip, etc.), and the electronic device may be the above-mentioned vehicle-mounted device.

[0227] As Figure 11 shown, the object inspection device 100 may include: a first acquisition module 110, a second acquisition module 120, and a prediction module 130.

[0228] The first acquisition unit 110 is configured to acquire the position information of the vehicle at the current moment;

[0229] The query unit 120 is configured to query the position information of the stationary objects near the vehicle in the first map according to the position information of the vehicle;

[0230] The determination unit 130 is configured to determine a first distance between the vehicle and the object according to the position information of the vehicle and the position information of the object;

[0231] The second acquisition unit 140 is configured to acquire a first image collected by the vehicle-mounted camera on the vehicle at the current moment when the first distance meets a preset distance threshold, and the first image includes the object, and the preset distance threshold is determined based on the height of the object;

[0232] The inspection unit 150 is configured to obtain an inspection result of the object according to the operation status information of the object in the first image.

[0233] In one embodiment, the inspection unit 150 is specifically configured to determine a position area of the object in the first image; detect the operation status information of the object within the position area; and obtain the inspection result of the object according to the operation status information of the object.

[0234] In one embodiment, the inspection unit 150 is specifically configured to convert the first image into a grayscale image; perform binarization processing on the grayscale image of the first image; and determine a position area of the object in the binarized first image according to the position information of the object.

[0235] In some embodiments, the target object is a street lamp, and the operating condition information of the target object includes the light spot area of the street lamp. The inspection unit 150 is specifically configured to determine that the inspection result of the street lamp is normal when the light spot area of the street lamp is greater than or equal to a preset light spot area; and determine that the inspection result of the street lamp is abnormal when the light spot area of the street lamp is less than the preset light spot area.

[0236] In some embodiments, if the first distance includes the longitudinal distance between the vehicle and the street lamp, the preset distance threshold includes a first longitudinal distance threshold and a second longitudinal distance threshold. The second acquisition unit 140 is specifically configured to acquire the first image collected by the in-vehicle camera at the current moment when it is determined that the longitudinal distance between the vehicle and the street lamp is greater than the first longitudinal distance threshold and less than the second longitudinal distance threshold.

[0237] In some embodiments, the first distance further includes the lateral distance between the vehicle and the street lamp, and the preset distance threshold further includes a lateral distance threshold. The second acquisition unit 140 is specifically configured to acquire the first image collected by the in-vehicle camera at the current moment when it is determined that the longitudinal distance between the vehicle and the street lamp is greater than the first longitudinal distance threshold and less than the second longitudinal distance threshold, and the lateral distance between the vehicle and the street lamp is less than the lateral distance threshold.

[0238] In some embodiments, the above-mentioned determination unit 130 is further configured to obtain the height information of the street lamp from the first map; and determine the first longitudinal distance threshold and the second longitudinal distance threshold according to the height information of the street lamp.

[0239] In some embodiments, the above-mentioned determination unit 130 is specifically configured to determine the first longitudinal distance threshold according to the height of the street lamp, a first preset value, and the upper viewing angle of the pitch angle of the in-vehicle camera; and determine the second longitudinal distance threshold according to the height of the street lamp, a second preset value, and the upper viewing angle of the pitch angle of the in-vehicle camera;

[0240] Wherein, both the first preset value and the second preset value are greater than 1, and the second preset value is greater than the first preset value.

[0241] In some embodiments, the above-mentioned determination unit 130 is specifically configured to determine the first longitudinal distance threshold according to the product of the height of the street lamp, the first preset value, and the trigonometric function value of the upper viewing angle. For example, the first longitudinal distance threshold is determined according to the following formula:

[0242] L1 = c1 * h1 * cotθ,

[0243] Wherein, L1 is the first longitudinal distance threshold, c1 is the first preset value, h1 is the height of the street lamp, and θ is the upper view angle of the pitch angle of the vehicle-mounted camera.

[0244] In some embodiments, the determining unit 130 is specifically configured to determine a second longitudinal distance threshold according to the product of the height of the street lamp, a second preset value, and the trigonometric function value of the upper view angle. For example, the second longitudinal distance threshold is determined according to the following formula:

[0245] L2 = c2 * h1 * cotθ,

[0246] Wherein, L2 is the first longitudinal distance threshold, c2 is the second preset value, h1 is the height of the street lamp, and θ is the upper view angle of the pitch angle of the vehicle-mounted camera.

[0247] In some embodiments, the determining unit 130 is further configured to determine a lateral distance threshold according to the longitudinal distance between the vehicle and the street lamp, a third preset value, and the horizontal view angle of the vehicle-mounted camera when it is determined that the longitudinal distance between the vehicle and the street lamp is greater than the first longitudinal distance threshold and less than the second longitudinal distance threshold, where the third preset value is a positive number less than 1.

[0248] In some embodiments, the determining unit 130 is specifically configured to determine a lateral distance threshold according to the product of the longitudinal distance between the vehicle and the street lamp, a third preset value, and the trigonometric function value of the horizontal view angle. For example, the lateral distance threshold is determined according to the following formula:

[0249] L3 = c3 * d1 * tanα,

[0250] Wherein, L3 is the lateral distance threshold, c3 is the third preset value, d1 is the longitudinal distance between the vehicle and the street lamp, and α is half of the horizontal view angle of the vehicle-mounted camera.

[0251] In some embodiments, the inspection unit 150 is further configured to send a first message when the inspection result of the street lamp is abnormal, and the first message includes the position information of the street lamp.

[0252] Optionally, the first message further includes the first image.

[0253] Optionally, the first map is a high-precision map.

[0254] It should be understood that the device embodiments and the method embodiments can correspond to each other, and similar descriptions can refer to the method embodiments. To avoid repetition, it will not be elaborated here. Specifically, Figure 11The device shown can execute the embodiments of the above method, and the foregoing and other operations and / or functions of each module in the device respectively implement the method embodiments corresponding to the encoder. For the sake of brevity, they will not be described herein again.

[0255] In the foregoing, the device of the embodiments of the present application has been described from the perspective of functional modules in combination with the drawings. It should be understood that the functional modules can be implemented in the form of hardware, or in the form of instructions in software, or in a combination of hardware and software modules. Specifically, the steps of the method embodiments in the present application can be completed by the integrated logic circuit in hardware in the processor and / or instructions in software form. The steps of the method disclosed in combination with the embodiments of the present application can be directly embodied as being executed and completed by a hardware decoding processor, or executed and completed by a combination of hardware and software modules in the decoding processor. Optionally, the software module can be located in a mature storage medium in the art such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, etc. This storage medium is located in the memory, and the processor reads the information in the memory and combines its hardware to complete the steps in the above method embodiments.

[0256] Figure 12 It is a block diagram of an electronic device related to the embodiments of the present application. The electronic device can be a vehicle-mounted device and is used to execute the target object inspection method described in the above embodiments. For specific reference, see the description in the above method embodiments.

[0257] As Figure 12 shown, the electronic device 30 may include: a memory 31 and a processor 32. The memory 31 is used to store a computer program 33 and transmit the program code 33 to the processor 32. In other words, the processor 32 can call and run the computer program 33 from the memory 31 to implement the method in the embodiments of the present application.

[0258] For example, the processor 32 can be used to execute the above method steps according to the instructions in the computer program 33.

[0259] In some embodiments of the present application, the processor 32 may include but is not limited to:

[0260] a general-purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, and so on.

[0261] In some embodiments of the present application, the memory 31 includes, but is not limited to:

[0262] A volatile memory and / or a non-volatile memory. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct rambus random access memory (DR RAM).

[0263] In some embodiments of the present application, the computer program 33 may be divided into one or more modules, and the one or more modules are stored in the memory 31 and executed by the processor 32 to complete the method for recording a page provided by the present application. The one or more modules may be a series of computer program instruction segments capable of performing specific functions, and the instruction segments are used to describe the execution process of the computer program 33 in the electronic device.

[0264] As Figure 12 shown, the electronic device 30 may further include:

[0265] A transceiver 34, and the transceiver 34 may be connected to the processor 32 or the memory 31.

[0266] Among them, the processor 32 may control the transceiver 34 to communicate with other devices. Specifically, it may send information or data to other devices, or receive information or data sent by other devices. The transceiver 34 may include a transmitter and a receiver. The transceiver 34 may further include an antenna, and the number of antennas may be one or more.

[0267] It should be understood that the various components in the electronic device 30 are connected through a bus system. Among them, the bus system includes not only a data bus, but also a power bus, a control bus, and a status signal bus.

[0268] According to one aspect of the present application, there is provided a computer storage medium having a computer program stored thereon. When the computer program is executed by a computer, the computer is enabled to execute the methods of the above method embodiments. Or rather, the embodiments of the present application further provide a computer program product containing instructions. When the instructions are executed by a computer, the computer is enabled to execute the methods of the above method embodiments.

[0269] According to another aspect of the present application, there is provided a computer program product or a computer program. The computer program product or the computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, enabling the computer device to execute the methods of the above method embodiments.

[0270] In other words, when implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions according to the embodiments of the present application are generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center in a wired manner (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or a wireless manner (such as infrared, wireless, microwave, etc.). The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or a data center that includes one or more integrated available media. The available medium may be a magnetic medium (such as a floppy disk, a hard disk, a magnetic tape), an optical medium (such as a digital video disc (DVD)), or a semiconductor medium (such as a solid state disk (SSD)), etc.

[0271] Those of ordinary skill in the art will appreciate that the modules and algorithm steps of each example described in connection with the embodiments disclosed herein can be implemented in electronic hardware, or in a combination of computer software and electronic hardware. Whether these functions are executed in hardware or software depends on the specific application and design constraints of the technical solution. A professional technician may use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of this application.

[0272] In several embodiments provided in this application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the modules is only a logical function division. In actual implementation, there may be other division methods. For example, multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed couplings or direct couplings or communication connections to each other can be through some interfaces. The indirect couplings or communication connections of devices or modules can be electrical, mechanical, or other forms.

[0273] The modules described as separate components may or may not be physically separated. The components shown as modules may or may not be physical modules, that is, they can be located in one place, or they can be distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. For example, in each embodiment of this application, the functional modules can be integrated into one processing module, or each module can exist physically alone, or two or more modules can be integrated into one module.

[0274] The above is only the specific implementation manner of this application, but the protection scope of this application is not limited thereto. Any person skilled in the art within the technical scope disclosed in this application can easily think of changes or substitutions, which should all be covered by the protection scope of this application. Therefore, the protection scope of this application should be subject to the protection scope of the claims.

Claims

1. A method for inspecting a target object, characterized in that, Including: Obtain the position information of the vehicle at the current moment; According to the position information of the vehicle, query the position information of the target objects in a stationary state near the vehicle in the first map; According to the position information of the vehicle and the position information of the target objects, determine the first distance between the vehicle and the target objects; When the first distance meets the preset distance threshold, obtain the first image collected by the on-vehicle camera installed on the vehicle at the current moment, where the first image includes the target objects, and the preset distance threshold is determined based on the height of the target objects; According to the operation status information of the target objects in the first image, obtain the inspection result of the target objects; When the first distance includes a longitudinal distance and a lateral distance, the preset distance threshold includes a longitudinal distance threshold and a lateral distance threshold, where the longitudinal distance threshold is determined based on the height of the target objects and the viewing angle of the on-vehicle camera, and the lateral distance threshold is determined based on the longitudinal distance threshold and the viewing angle of the on-vehicle camera.

2. The method according to claim 1, wherein The obtaining the inspection result of the target objects according to the operation status information of the target objects in the first image includes: Determine the position area of the target objects in the first image; Detect the operation status information of the target objects within the position area; According to the operation status information of the target objects, obtain the inspection result of the target objects.

3. The method according to claim 2, wherein The determining the position area of the target objects in the first image includes: Convert the first image into a grayscale image; Perform binarization processing on the grayscale image of the first image; According to the position information of the target objects, determine the position area of the target objects in the binarized first image.

4. The method according to claim 1, wherein The target objects are street lamps, and the operation status information of the target objects includes the spot area of the street lamps. The obtaining the inspection result of the target objects according to the operation status information of the target objects in the first image includes: When the spot area of the street lamp is greater than or equal to the preset spot area, determine that the inspection result of the street lamp is normal; When the spot area of the street lamp is less than the preset spot area, determine that the inspection result of the street lamp is abnormal.

5. The method according to claim 4, characterized in that, If the first distance includes the longitudinal distance between the vehicle and the street lamp, and the preset distance threshold includes a first longitudinal distance threshold and a second longitudinal distance threshold, then when the first distance meets the preset distance threshold, obtaining the first image collected by the on-vehicle camera installed on the vehicle at the current moment includes: When it is determined that the longitudinal distance between the vehicle and the street lamp is greater than the first longitudinal distance threshold and less than the second longitudinal distance threshold, obtain the first image collected by the on-vehicle camera at the current moment.

6. The method according to claim 5, characterized in that, The first distance further includes the lateral distance between the vehicle and the street lamp, the preset distance threshold further includes a lateral distance threshold, and when it is determined that the longitudinal distance between the vehicle and the street lamp is greater than the first longitudinal distance threshold and less than the second longitudinal distance threshold, obtaining the first image collected by the on-vehicle camera at the current moment includes: When it is determined that the longitudinal distance between the vehicle and the street lamp is greater than the first longitudinal distance threshold and less than the second longitudinal distance threshold, and the lateral distance between the vehicle and the street lamp is less than the lateral distance threshold, obtain the first image captured by the in-vehicle camera at the current moment.

7. The method according to claim 6, characterized in that, The method further includes: Obtain the height information of the street lamp from the first map, where the first map includes the height information of the street lamp; Determine the first longitudinal distance threshold and the second longitudinal distance threshold according to the height information of the street lamp.

8. The method according to claim 7, wherein The determining the first longitudinal distance threshold and the second longitudinal distance threshold according to the height information of the street lamp includes: Determine the first longitudinal distance threshold according to the height of the street lamp, a first preset value, and the upper viewing angle of the pitch angle of the in-vehicle camera; Determine the second longitudinal distance threshold according to the height of the street lamp, a second preset value, and the upper viewing angle of the pitch angle of the in-vehicle camera; Wherein, both the first preset value and the second preset value are greater than 1, and the second preset value is greater than the first preset value.

9. The method according to claim 8, wherein The determining the first longitudinal distance threshold according to the height of the street lamp, a first preset value, and the upper viewing angle of the pitch angle of the in-vehicle camera includes: Determine the first longitudinal distance threshold according to the product of the height of the street lamp, the first preset value, and the trigonometric function value of the upper viewing angle.

10. The method according to claim 8, characterized in that The determining the second longitudinal distance threshold according to the height of the street lamp, a second preset value, and the upper viewing angle of the pitch angle of the in-vehicle camera includes: Determine the second longitudinal distance threshold according to the product of the height of the street lamp, the second preset value, and the trigonometric function value of the upper viewing angle.

11. The method according to claim 6, wherein The method further includes: When it is determined that the longitudinal distance between the vehicle and the street lamp is greater than the first longitudinal distance threshold and less than the second longitudinal distance threshold, determine the lateral distance threshold according to the longitudinal distance between the vehicle and the street lamp, a third preset value, and the horizontal viewing angle of the in-vehicle camera, where the third preset value is a positive number less than 1.

12. The method according to claim 11, wherein The determining the lateral distance threshold according to the longitudinal distance between the vehicle and the street lamp, a third preset value, and the horizontal viewing angle of the in-vehicle camera includes: Determine the lateral distance threshold according to the product of the longitudinal distance between the vehicle and the street lamp, the third preset value, and the trigonometric function value of the horizontal viewing angle.

13. A target inspection device, characterized in that, Includes: A first acquisition unit for acquiring the position information of the vehicle at the current moment; A query unit for querying the position information of the stationary target near the vehicle in the first map according to the position information of the vehicle; A determination unit for determining the first distance between the vehicle and the target according to the position information of the vehicle and the position information of the target; A second acquisition unit for acquiring the first image captured by the in-vehicle camera mounted on the vehicle at the current moment when the first distance satisfies a preset distance threshold, where the first image includes the target, and the preset distance threshold is determined based on the height of the target; An inspection unit, configured to obtain an inspection result of the target object according to the operation status information of the target object in the first image; When the first distance includes a longitudinal distance and a lateral distance, the preset distance threshold includes a longitudinal distance threshold and a lateral distance threshold, wherein the longitudinal distance threshold is determined based on the height of the target object and the viewing angle of the vehicle-mounted camera, and the lateral distance threshold is determined based on the longitudinal distance threshold and the viewing angle of the vehicle-mounted camera.

14. An electronic device, characterized in that, Comprising: A memory and a processor; The memory is configured to store a computer program; The processor is configured to execute the computer program to implement the method according to any one of claims 1 to 12 above.

15. A computer-readable storage medium, characterized in that, Computer-executable instructions are stored in the computer-readable storage medium, and when the computer-executable instructions are executed by the processor, they are used to implement the method according to any one of claims 1 to 12.

Images