Scene data acquisition method and device, storage medium and electronic device

Abstract

The embodiment of the invention provides a scene data collection method and device, a storage medium and an electronic device, and the method comprises the steps: obtaining and collecting image data of a front-view camera of a vehicle through a browser, and obtaining first front-view image data; performing scene detection on the first front-view image data by using a target detection model operated by the browser to obtain a first detection result, the target detection model being a model for detecting a group of preset scenes, and the first detection result being used for indicating a preset scene detected from the first front-view image data in the group of preset scenes; and under the condition that a preset scene detected from the first front view image data comprises a specified scene, sending a first acquisition request to an acquisition vehicle through the browser, the first acquisition request being used for requesting the acquisition vehicle to acquire scene data, and storing the acquired scene data. According to the invention, the problems of high labor cost and low data acquisition efficiency of a scene data acquisition method in the prior art are solved.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of data processing, and specifically, to a method and device for collecting scene data, a storage medium, and an electronic device. Background Art

[0002] When collecting data for autonomous driving, a collection vehicle needs to be equipped with a driver and a collector. When multiple collection vehicles are used for data collection, a lot of manpower costs are required. For special scene collection, such as pedestrians, cones, special vehicles, etc., the only way is for the collector to observe and manually click on the data collection.

[0003] It can be seen that the scene data collection method in the related technology has the problems of high labor cost and low data collection efficiency. Summary of the invention

[0004] The embodiments of the present application provide a scene data collection method and device, a storage medium, and an electronic device to at least solve the problems of high labor cost and low data collection efficiency in the scene data collection method in the related art.

[0005] According to one aspect of an embodiment of the present application, a method for collecting scene data is provided, comprising: obtaining image data of a front-view camera of a collection vehicle through a browser to obtain first front-view image data; performing scene detection on the first front-view image data using a target detection model run by the browser to obtain a first detection result, wherein the target detection model is a model for detecting a group of preset scenes, and the first detection result is used to indicate a preset scene detected from the first front-view image data in the group of preset scenes; in a case where the preset scene detected from the first front-view image data includes a specified scene, sending a first collection request to the collection vehicle through the browser, wherein the first collection request is used to request the collection vehicle to collect scene data and save the collected scene data.

[0006] According to another aspect of an embodiment of the present application, a scene data collection device is also provided, including: a first obtaining unit, used to obtain image data of a front-view camera of a collection vehicle through a browser to obtain first front-view image data; a second obtaining unit, used to perform scene detection on the first front-view image data using a target detection model run by the browser to obtain a first detection result, wherein the target detection model is a model for detecting a group of preset scenes, and the first detection result is used to indicate a preset scene detected from the first front-view image data in the group of preset scenes; a first sending unit, used to send a first collection request to the collection vehicle through the browser when the preset scene detected from the first front-view image data includes a specified scene, wherein the first collection request is used to request the collection vehicle to collect scene data and save the collected scene data.

[0007] According to another aspect of the embodiments of the present application, a computer-readable storage medium is provided, in which a computer program is stored, wherein the computer program is configured to execute the steps of any of the above method embodiments when run.

[0008] According to another aspect of the embodiments of the present application, an electronic device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0009] According to another aspect of the embodiments of the present application, a computer program product or a computer program is also provided, the computer program product or the computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the steps in any of the above method embodiments.

[0010] Through the embodiment of the present application, a target detection model is used on the browser side to perform scene detection, and when a specific scene is detected, a collection vehicle is controlled to collect scene data. First front view image data is obtained by obtaining image data of a front view camera of the collection vehicle through a browser; a target detection model running on the browser is used to perform scene detection on the first front view image data to obtain a first detection result, wherein the target detection model is a model for detecting a set of preset scenes, and the first detection result is used to indicate a preset scene detected from the first front view image data in a set of preset scenes; when the preset scene detected from the first front view image data includes a specified scene, the browser is used to send a detection result to the collection vehicle. The first acquisition request is used to request the acquisition vehicle to collect scene data and save the collected scene data. Since the target machine detection model running on the browser performs scene detection on the image data of the front-view camera of the acquisition vehicle, and controls the acquisition vehicle to collect scene data when a specified scene (a scene requiring scene data collection) is detected, the purpose of automatically collecting the driving data of the acquisition vehicle remotely can be achieved. Since there is no need to configure a collector for each acquisition vehicle, the technical effect of reducing labor costs and improving data collection efficiency can be achieved, thereby solving the problems of high labor costs and low data collection efficiency in the scene data collection method in the related technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 is a schematic diagram of an application environment of an optional scene data collection method according to an embodiment of the present invention;

[0012] Figure 2 is a flow chart of an optional scene data collection method according to an embodiment of the present application;

[0013] Figure 3 is a schematic diagram of an optional scene data collection method according to an embodiment of the present application;

[0014] Figure 4 is a schematic diagram of another optional scene data collection method according to an embodiment of the present application;

[0015] Figure 5 is a schematic diagram of another optional scene data collection method according to an embodiment of the present application;

[0016] Figure 6 is a structural block diagram of an optional scene data collection device according to an embodiment of the present application;

[0017] Figure 7 It is a structural block diagram of a computer system of an optional electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0018] In order to enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only embodiments of a part of the present application, not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work (for example, feature splitting in the same embodiment and combining features in different embodiments, if there is no contradiction) shall fall within the scope of protection of the present application.

[0019] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0020] According to one aspect of an embodiment of the present application, a method for collecting scene data is provided. Optionally, in this embodiment, the above-mentioned method for collecting scene data can be applied to Figure 1 The hardware scenario shown includes a vehicle end 102, a server end 104 and a terminal device 106. Figure 1 As shown, the vehicle side 102, the server side 104 and the terminal device 106 can be connected through a network and exchange information. The vehicle side 102 can be a collection vehicle for scene data collection or a vehicle-mounted terminal on a collection vehicle. The server side 104 can be a single server or a server cluster, or a combination of different types of servers. The terminal device 106 can be a desktop computer, a laptop computer, a portable terminal or other devices that can display the content required to be displayed by the browser side.

[0021] The above network may include but is not limited to at least one of the following: wired network, wireless network. The above wired network may include but is not limited to at least one of the following: wide area network, metropolitan area network, local area network, and the above wireless network may include but is not limited to at least one of the following: WIFI (Wireless Fidelity), Bluetooth, etc., and may also be other networks, which are not limited in this embodiment.

[0022] Optionally, the scene data collection method in this embodiment may be executed by the terminal device 106, or may be executed by the terminal device 106 in conjunction with at least one of the vehicle end 102 and the server end 104. Taking the execution by the terminal device 106 as an example, Figure 2 is a flow chart of an optional scene data collection method according to an embodiment of the present application, such as Figure 2 As shown, the above method includes:

[0023] Step S202, acquiring image data collected by a front-view camera of the vehicle through a browser to obtain first front-view image data;

[0024] Step S204, performing scene detection on the first front view image data using the target detection model run by the browser to obtain a first detection result, wherein the target detection model is a model used to detect a set of preset scenes, and the first detection result is used to indicate a preset scene detected from the first front view image data in the set of preset scenes;

[0025] Step 206, when the preset scene detected from the first forward-view image data includes the designated scene, a first acquisition request is sent to the acquisition vehicle through the browser, wherein the first acquisition request is used to request the acquisition vehicle to acquire scene data and save the acquired scene data.

[0026] The scene data collection method in this embodiment can be applied to the field of autonomous driving data collection, and can be applied to scene data collection of special scenes through collection vehicles. In autonomous driving data collection, a collection vehicle usually needs to be equipped with a driver and a collector, and the collection personnel will process and analyze the collected data. When multiple collection vehicles are used for collection, a corresponding number of collectors are usually required, which requires a lot of manpower costs. For special scenes, such as special vehicles, pedestrians, cone barrels and other scenes, the only way is for the collector to observe and manually click and mark. Due to the influence of manual operation, the collected data may contain errors and the collection efficiency is low.

[0027] In order to at least partially solve the above technical problems, in this embodiment, the target machine detection model running on the browser side performs scene detection on the image data of the front-view camera of the collection vehicle, and controls the collection vehicle to collect scene data when a specified scene (a scene that requires scene data collection, such as the special scene mentioned above) is detected. This can achieve the purpose of remote automatic collection of driving data of the collection vehicle. Since there is no need to configure a collector for each collection vehicle, the labor cost can be reduced and the data collection efficiency can be improved.

[0028] For the browser side, the browser side can obtain the image data of the front-view camera of the acquisition vehicle to obtain the first front-view image data. The image data of the front-view camera of the acquisition vehicle can be obtained from the vehicle side, where the vehicle side can refer to the acquisition vehicle itself. In this case, the acquisition vehicle has network communication capabilities (i.e., the acquisition vehicle is integrated with a network communication module); the vehicle side can also refer to the vehicle-mounted terminal set on the acquisition vehicle, through which the acquisition vehicle can be represented for network communication.

[0029] The image data of the front-view camera of the vehicle can be obtained through the server side, and correspondingly, a communication connection is established between the browser side and the vehicle side. When obtaining the front-view image of the vehicle side, in order to facilitate data transmission, the transmission of the image data of the front-view camera of the vehicle can be completed based on Kafka. Here, Kafka is a message subscription-publishing system, including a message producer (Producer) and a message consumer (Consumer). The Producer pushes the message to the Kafka cluster, and the Consumer pulls the message from the Kafka cluster. Kafka classifies messages in terms of Topic (topic). Each Topic can be considered as a type of message, and different Topics are independent of each other. Correspondingly, the front-view camera image data and CAN (Controller Area Network bus) bus information (which may include geographic positioning, steering information, etc.) can be subscribed in the vehicle-side ROS (Robot Operating System) service, and the subscribed data will be uploaded to the Kafka server. The browser or backend server can consume the image data in the Kafka server to obtain the image data collected by the vehicle's front-view camera.

[0030] It should be noted that the front-view camera of the vehicle can be a camera that can capture images within a certain area in front of the vehicle, and it can be located on the left side, right side, top side or other positions of the vehicle; CANBUS information refers to data transmitted between different control systems and devices inside the vehicle through the CANBUS protocol, which may include but is not limited to at least one of the following: engine information (engine speed, engine temperature, engine load, etc.), vehicle speed information (vehicle speed, acceleration, deceleration, etc.), safety information (seat belt status, door switch status, airbag status, etc.), lighting information (light status, brightness adjustment, etc.), instrument information (fuel level, cruising range, oil pressure, etc.), location information (i.e., geolocation); Kafka server is a distributed stream processing platform, its high throughput ensures that data can be transmitted in real time and efficiently, and low latency ensures the real-time nature of the data.

[0031] It should also be noted that the browser side can be located on the aforementioned terminal device, and the background server can be a background server that communicates with the browser side. The matching here can be based on the identification information of the browser itself, or it can be based on the user object of the browser side (observer or other relevant user), as long as it can be guaranteed that the relevant user can obtain the image data of the front-view camera of the vehicle through the browser side.

[0032] Optionally, in this embodiment, the browser side acquires the image data of the front-view camera of the vehicle in a continuous process. For example, the image data of the front-view camera of the vehicle can be acquired periodically. The acquisition period can be 5s, 10s or other periods. The first front-view image data is the image data acquired at a certain time. The browser side acquires the image data of the front-view camera of the vehicle in an event-triggered process. The first front-view image data can be the image data acquired after a certain event is triggered. In this embodiment, the form of the browser side acquiring the image data of the front-view camera of the vehicle is not limited.

[0033] In order to collect scene data for the required scene, a group of scenes to be detected can be preset, that is, a group of preset scenes (for example, special scenes), and scene detection can be performed on the first front view image data to obtain a first detection result, and the first detection result is used to indicate the preset scene detected from the first front view image data in the group of preset scenes. The preset scene can be a preset scene target, which can include but is not limited to at least one of the following: traffic participants, traffic sign facilities, where traffic participants can be vehicles (for example, special vehicles), pedestrians and other targets participating in traffic, and traffic sign facilities are facilities that serve as traffic safety warnings, which can be cones, water barriers, etc.

[0034] Optionally, scene detection of the first forward-looking image data may be performed using a trained target detection model, where the target detection model may be a multi-task target detection model (a detection model capable of detecting a set of preset scenes), or may include multiple detection models for detecting different preset scenes. For the forward-looking camera image data, the multi-task target detection model may be a forward-looking multi-task target detection model (which may be a target detection model capable of detecting special scenes such as vehicles, pedestrians, cones, water barriers, etc.). The target detection model may be obtained by training an initial detection model using training image data with preset scenes labeled.

[0035] The target detection model can be run by the browser, and the browser can load and run the target detection model (for example, the aforementioned forward-looking multi-task target detection model). If the browser cannot directly run the target detection model (for example, the target detection model running on the server cannot be directly run by the browser due to compatibility issues), the target detection model can be compiled into onnx (Open Neural Network Exchange) format or other formats that can be run by the browser through PyTorch (an open source machine learning library). Correspondingly, TensorFlow.js (an open source machine learning library) can be used to load and run the aforementioned onnx format target detection model.

[0036] If the preset scene detected from the first forward-view image data includes a specified scene, the browser side can send a first collection request to the collection vehicle to request the collection vehicle to collect scene data and save the collected scene data. After receiving the first collection request, the collection vehicle can perform scene data collection, and the collected scene data can be saved to a set storage location, for example, stored in a specified database or storage system. The collection time of each collection on the vehicle side can be pre-set, for example, 1 minute, 2 minutes or other values.

[0037] It should be noted that all preset scenes in a set of preset scenes may be designated scenes. In this case, if a preset scene is detected from the first front-view image data, a first acquisition request can be sent to the collection vehicle; only some of the preset scenes in a set of preset scenes may be designated scenes. In this case, the preset scenes detected from the first front-view image data can be traversed to determine whether a designated scene exists. The vehicle for collecting scene data may be the aforementioned front-view camera, or a radar sensor or other collection component capable of collecting scene data. The aforementioned front-view camera or other collection component can be kept in an on state, and the image data within its field of view can be transmitted to the aforementioned browser or intermediate device (for example, a kafka server). At this time, the image data is only an image data stream (will not be saved or stored on disk). After receiving the collection instruction, the scene data obtained by the collection vehicle for data collection will be saved (stored on disk).

[0038] For example, when the browser determines that a special scene (i.e., a specified scene) selected by the user is detected from the image data of the front-view camera, it can send a collection request to the vehicle. After receiving the collection request, the vehicle executes the collection (the collection time is 1 minute) and saves the collected scene data.

[0039] Here, autonomous driving data is collected remotely in a web browser, and visual detection algorithms (i.e., target detection models) are executed in the browser to automatically collect data for special scenarios. When multiple collection vehicles are collecting data at the same time, online data collection can be controlled in a web browser, without the need for collectors to be assigned to each vehicle, which greatly saves data collection costs. At the same time, for special scene collection, visual algorithms can be executed in a web browser to achieve automatic collection, saving server resources and improving collection efficiency.

[0040] Through the above steps, the image data of the front-view camera of the acquisition vehicle is obtained through the browser to obtain the first front-view image data, and the target detection model run by the browser is used to perform scene detection on the first front-view image data to obtain a first detection result, wherein the target detection model is a model for detecting a set of preset scenes, and the first detection result is used to indicate a preset scene detected from the first front-view image data in a set of preset scenes. When the preset scene detected from the first front-view image data includes a specified scene, a first acquisition request is sent to the acquisition vehicle through the browser, wherein the first acquisition request is used to request the acquisition vehicle to collect scene data and save the collected scene data, which can solve the problems of high labor cost and low data acquisition efficiency in the scene data acquisition method in the related art, improve the acquisition efficiency, and reduce the labor cost.

[0041] In an exemplary embodiment, obtaining the image data of the front-view camera may be achieved through the topic subscription and consumption function of Kafka. Correspondingly, obtaining the image data of the front-view camera of the vehicle through the browser may be obtaining the image data of the front-view camera of the vehicle consumed from the Kafka server through the browser. Obtaining the image data of the front-view camera may be triggered by the user object of the browser by performing a play trigger operation on the browser.

[0042] Correspondingly, the image data of the front-view camera of the vehicle is acquired through the browser to obtain the first front-view image data, including:

[0043] S11, in response to the detected play trigger operation, sending a play request to the first server through the browser, wherein the play request is used to request to play the image data of the front-view camera;

[0044] S12, receiving the image data of the front-view camera returned by the first server in response to the play request, and obtaining first front-view image data, wherein the image data of the front-view camera is obtained by the first server by consuming the image data of the front-view camera in the second server, and the second server is a Kafka server.

[0045] The user of the browser can trigger the browser to obtain the image data of the front-view camera of the vehicle by clicking the play button on the browser or performing other play triggering operations. In response to the detected play triggering operation, the browser can send a play request to the first server, and the play request is used to request to play the image data of the front-view camera. The first server can be the aforementioned background server. In response to the received play request, the first server can consume the image data of the front-view camera in the second server (i.e., the Kafka server) and transmit the consumed image data of the front-view camera to the browser. At this time, the browser can obtain the first front-view image data.

[0046] After the first forward-view image data is acquired, the first forward-view image data can be directly played on the browser. Optionally, in order to facilitate the user of the browser to determine the specified scene in the first forward-view image data, after the target detection model running in the browser is used to perform scene detection on the first forward-view image data and a first detection result is obtained, the method further includes:

[0047] S13, playing the target image data on the browser, wherein the target image data is obtained by adding a scene identifier for a designated scene to the first front-view image data.

[0048] Here, the data played on the browser is the target image data, which is obtained by adding a scene identifier for the specified scene on the first front view image data. The scene identifier may include but is not limited to at least one of the following: a bounding box, a label. Playing the target image data may be achieved by first displaying the first front view image data, and then superimposing the specified scene and adding the scene identifier on the specified scene in the displayed first front view image data, or by first adding the specified scene and adding the scene identifier on the specified scene in the first front view image data, and then playing the obtained target image data. Optionally, the scene identifiers required by different types of specified scenes may be at least partially different.

[0049] For example, a user clicks the play button on a page (such as Figure 3 As shown in the figure, the front-end (browser) sends a play message to the back-end server through websocket (network socket); the back-end service consumes the image data in the kafka server and sends the image data to the front-end through websocket; the front-end performs target detection on the received image data by running the forward-looking multi-task target detection model, obtains the detection result, and displays the recognition result in the browser, such as the bounding box and label of the object detection, as shown in the figure. Figure 3 shown.

[0050] The detection result format may be in JSON (a lightweight data exchange format). An example of the detection result is as follows:

[0051]

[0052]

[0053]

[0054]

[0055] Through this embodiment, the transmission of the front-view camera image data based on Kafka can improve the flexibility of data transmission; the acquisition of the front-view camera image data based on the triggered playback operation can improve the effectiveness of the acquisition of the front-view camera image data and avoid unnecessary acquisition operations.

[0056] In an exemplary embodiment, a target detection model running in a browser is used to perform scene detection on the first front-view image data to obtain a first detection result, including:

[0057] S21, inputting the first front view image data into the target detection model, and using the network assembly accelerator to perform model reasoning of the target detection model to obtain a first detection result.

[0058] In this embodiment, a WebAssembly accelerator can be used to increase the speed of model reasoning of the target detection model. Here, the WebAssembly accelerator is used to accelerate the calculation by using the multi-threading and single instruction multiple data stream instruction set of the browser. Correspondingly, the first front view image data can be input into the target detection model, and the model reasoning of the target detection model can be performed by using the WebAssembly accelerator to obtain a first detection result.

[0059] For example, a WebAssembly accelerator can be used to perform model reasoning. WebAssembly can use the browser's multi-threading and SIMD (Single Instruction Multiple Data) instruction set to accelerate calculations.

[0060] Through this embodiment, the network assembly accelerator is used to perform model reasoning, which can increase the speed of model reasoning and further improve the efficiency of scene detection.

[0061] In an exemplary embodiment, the specified scene may be a preset scene selected from a set of preset scenes, and the selection operation may be performed by the user of the browser before performing scene detection on the first front view image data using the object detection model. Correspondingly, after performing scene detection on the first front view image data using the object detection model running in the browser to obtain the first detection result, the method further includes:

[0062] S31, traversing the first detection result until a traversal end condition is met, wherein the traversal end condition includes: traversing to a specified scene; and the traversal of the first detection result is completed.

[0063] In this embodiment, considering that scene data collection needs to be performed regardless of whether one or more designated scenes are detected, if it is determined that there is a designated scene in the preset scenes detected from the first front view image data, a collection request can be sent, and the subsequent process of determining whether the designated scene exists is terminated. Correspondingly, after obtaining the first detection result, the first detection result can be traversed until the traversal end condition is met, and the traversal end condition includes traversing to the designated scene and the first detection result has been traversed. Here, as long as any one of the traversal end conditions is met, the traversal can be terminated.

[0064] For example, the browser obtains the front view image and detects image elements. According to the special scenes selected by the user, the detection results are traversed. If the detection results contain the special scenes selected by the user, a collection request is sent to the server through websocket. After receiving the request, the server sends the collection request to the vehicle through kafka.

[0065] Here, an example of configuring the collection scenario and various parameters in the web application can be as follows Figure 4 As shown in the figure, in addition to configuring the collected vehicles, collectors, collection areas, environmental parameters, time, and storage location of the collected scene data, for multi-vehicle collection scenarios, you can also configure the collection mode and master-slave topic. In addition, you can also configure object detection trigger conditions, CANBUS trigger conditions, and scene classification trigger conditions to adapt to different collection requirements. The schematic diagram of multi-vehicle collection can be as follows Figure 5 shown.

[0066] Through this embodiment, whether a specified scene is detected is determined by traversing the detection results, and the traversal is terminated when a specified scene is traversed, so that the efficiency of data traversal can be improved.

[0067] In an exemplary embodiment, the image data of the front-view camera is continuously acquired, that is, the browser can continuously acquire the image data of the front-view camera; the acquisition time of each scene data acquisition by the acquisition vehicle is a specified time. Each time a specified scene is detected from the acquired image data of the front-view camera, a collection request is sent to the collection vehicle, and each time the collection vehicle receives the collection request, the time of the scene data acquisition is refreshed.

[0068] Optionally, in order to reduce the occupation of network resources and taking into account the continuity of the appearance of the specified scene (the same specified scene may appear in the image data of the continuous forward-looking camera), the interval time between the current moment and the moment when the acquisition request was sent last time may be greater than or equal to the specified interval time, that is, the acquisition request is sent again when the interval time condition is met. The interval time condition is that the interval time between the time when the acquisition request was sent last time is greater than or equal to the specified interval time. The interval time condition can be equal to the specified time. Taking into account the delay in the transmission of the acquisition request and avoiding the omission of scene data, the specified interval time can also be less than the specified time. In this case, when special scenes are detected continuously, the vehicle side will perform continuous acquisition.

[0069] Correspondingly, after acquiring the image data of the front-view camera of the vehicle through the browser to obtain the first front-view image data, the above method further includes:

[0070] S41, acquiring image data of a front-view camera of a vehicle through a browser to obtain second front-view image data;

[0071] S42, performing scene detection on the second front view image data using the target detection model to obtain a second detection result, wherein the second detection result is used to indicate a preset scene detected from the second front view image data in a group of preset scenes;

[0072] S43, when the preset scene detected from the second forward-view image data includes the specified scene and the interval between the current moment and the sending moment of the first collection request satisfies the interval time condition, sending a second collection request to the collection vehicle through the browser, wherein the second collection request is used to request the collection vehicle to collect scene data and save the collected scene data;

[0073] After the first forward-view image data, the browser can obtain the image data of the forward-view camera of the acquisition vehicle, that is, the second forward-view image data, and perform scene detection in a manner similar to that in the aforementioned embodiment to obtain a second detection result, and the second detection result is used to indicate a preset scene detected from the second forward-view image data in a set of preset scenes. If the preset scene detected from the second forward-view image data includes a specified scene, and the interval time between the current moment and the sending moment of the first acquisition request meets the interval time condition, the browser can send a second acquisition request to the acquisition vehicle to request the acquisition vehicle to collect scene data and save the collected scene data. The method of sending the second acquisition request is similar to the method of sending the first acquisition request, and will not be repeated here.

[0074] Here, determining whether the preset scene detected from the second forward-view image data includes the specified scene and determining whether the interval time between the current moment and the sending moment of the first acquisition request satisfies the interval time condition may be performed simultaneously or successively. For example, it may be determined first whether the preset scene detected from the second forward-view image data includes the specified scene, and then whether the interval time between the current moment and the sending moment of the first acquisition request satisfies the interval time condition; or, it may be determined first whether the interval time between the current moment and the sending moment of the first acquisition request satisfies the interval time condition, and then whether the preset scene detected from the second forward-view image data includes the specified scene.

[0075] Through this embodiment, by setting the interval time condition, the collection request will be sent again only when the interval time between the current moment and the sending time of the previous collection request is greater than or equal to the specified interval time. This can reduce the number of collection requests sent, reduce the occupancy of network resources, and at the same time improve the continuity of scene data collection.

[0076] In an exemplary embodiment, the method further comprises:

[0077] S51, continuously acquiring the coordinate information of the collected vehicle through the browser to obtain a set of coordinate information of the collected vehicle;

[0078] S52, determining the actual driving trajectory of the collected vehicle based on the coordinate information set of the collected vehicle;

[0079] S53, when the trajectory deviation between the actual driving trajectory and the expected driving trajectory is greater than or equal to the preset deviation threshold, a deviation prompt message is issued through the browser to prompt the collection vehicle to deviate from the expected driving trajectory, wherein the expected driving trajectory is a driving trajectory specified for the collection vehicle based on the preset starting position point and the end position point.

[0080] In order to facilitate monitoring of the driving route of the collected vehicle and improve the accuracy of data collection, the browser can continuously obtain the coordinate information of the collected vehicle to obtain the coordinate information set of the collected vehicle; and determine the actual driving trajectory of the collected vehicle based on the coordinate information set of the collected vehicle. Here, the actual driving trajectory of the collected vehicle can be determined based on all the coordinate information in the coordinate information set of the collected vehicle. Considering the density of the coordinate information of the collected vehicle, the actual driving trajectory of the collected vehicle can be determined based on only part of the coordinate information in the coordinate information set of the collected vehicle.

[0081] If the trajectory deviation between the actual driving trajectory and the expected driving trajectory is greater than or equal to the preset deviation threshold, a deviation prompt message is issued through the browser to prompt the collection vehicle to deviate from the expected driving trajectory. Among them, the trajectory deviation between the actual driving trajectory and the expected driving trajectory can be determined based on the entire actual driving trajectory, or it can be determined based on the coordinate information at a certain moment. Issuing the deviation prompt message through the browser can be to directly display the deviation prompt message through the browser, or it can be to send the deviation prompt message to the collection vehicle through the browser. The expected driving trajectory is the driving trajectory specified for the collection vehicle based on the preset starting position point and the ending position point. Here, the starting position point and the ending position point can be set on the third-party application map to generate the vehicle driving trajectory, and map it to the tile map to obtain the expected driving trajectory. Here, the actual driving trajectory is generated based on the same tile map.

[0082] For example, the vehicle's driving trajectory can be monitored. The backend server consumes the CANBUS data in Kafka and sends it to the frontend through websocket. The frontend saves the vehicle's GPS (Global Positioning System) trajectory and determines the trajectory deviation between the vehicle's GPS trajectory and the set driving trajectory. If the trajectory deviation is greater than the threshold, the WEB browser will issue a deviation reminder.

[0083] Through this embodiment, the accuracy of data collection can be improved by monitoring the driving trajectory of the vehicle and issuing a deviation reminder when the trajectory deviation between the actual driving trajectory of the vehicle and the expected driving trajectory is greater than the deviation threshold.

[0084] In an exemplary embodiment, the method further comprises:

[0085] S61, performing simplification tolerance processing on the coordinate information set of the collected vehicle according to preset simplification parameters to obtain simplified coordinate information of the collected vehicle;

[0086] S62, based on the simplified coordinate information of the collected vehicle, the vehicle position and the driving track of the collected vehicle are displayed in real time on the browser.

[0087] In this embodiment, the front end can display the current vehicle position in real time. In order to avoid the browser crash caused by the vehicle coordinate information being too dense, the coordinate information set of the collected vehicle can be simplified by the tolerance processing to simplify the coordinate information set of the collected vehicle. Here, the simplification tolerance specifies the minimum distance or angle that can be ignored in the simplification process. By setting the simplification tolerance, the number of nodes of the element can be reduced, thereby simplifying the geometric shape. The basic principle of the simplification tolerance algorithm is: according to the set tolerance value, it is determined whether the nodes in the element can be deleted or merged. If the distance between two nodes is less than the tolerance value, or the impact on the overall shape of the element is small, these nodes can be regarded as redundant and can be deleted or merged. When setting the tolerance value, the larger the tolerance value, the higher the degree of simplification, but more detailed information may be lost; the smaller the tolerance value, the lower the degree of simplification, but more detailed information can be retained.

[0088] For the coordinate information set of the collected vehicle, the coordinate information set of the collected vehicle can be simplified and processed with tolerance according to preset simplification parameters to obtain simplified coordinate information of the collected vehicle; based on the simplified coordinate information of the collected vehicle, the vehicle position and driving trajectory of the collected vehicle are displayed in real time on the browser.

[0089] For example, when a vehicle starts to move, the front end will simplify the set of GPS points by simplifying the tolerance (the simplification parameter can be 0.0001) and ensure the vehicle's GPS track to avoid too dense GPS points causing the web browser to crash. In this way, remote collection can observe the vehicle's location and driving track in real time.

[0090] Through this embodiment, by simplifying the tolerance processing of the coordinate information set of the collected vehicle, it is possible to reduce the amount of coordinate information that the browser needs to process while ensuring the effectiveness of the driving trajectory display, thereby avoiding browser crashes.

[0091] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods of each embodiment of the present application.

[0092] According to another aspect of the embodiments of the present application, a scene data acquisition device is also provided, which is used to implement the scene data acquisition device provided in the above embodiments, and will not be repeated hereafter. As used below, the term "module" can implement a combination of software and / or hardware of a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0093] Figure 6 is a structural block diagram of an optional scene data collection device according to an embodiment of the present application, such as Figure 6 As shown, the device comprises:

[0094] A first acquisition unit 602 is used to acquire image data of a front-view camera of a vehicle through a browser to obtain first front-view image data;

[0095] A first detection unit 604 is used to perform scene detection on the first front view image data using the target detection model run by the browser to obtain a first detection result, wherein the target detection model is a model used to detect a set of preset scenes, and the first detection result is used to indicate a preset scene detected from the first front view image data in the set of preset scenes;

[0096] The first sending unit 606 is used to send a first acquisition request to the acquisition vehicle through the browser when the preset scene detected from the first forward-view image data includes a specified scene, wherein the first acquisition request is used to request the acquisition vehicle to acquire scene data and save the acquired scene data.

[0097] It should be noted that the first acquisition unit 602 in this embodiment can be used to execute the above step S202, the first detection unit 604 in this embodiment can be used to execute the above step S204, and the first sending unit 606 in this embodiment can be used to execute the above step S206.

[0098] Through the embodiments of the present application, image data of a front-view camera of a collection vehicle is obtained through a browser to obtain first front-view image data, and a target detection model run by the browser is used to perform scene detection on the first front-view image data to obtain a first detection result, wherein the target detection model is a model for detecting a set of preset scenes, and the first detection result is used to indicate a preset scene detected from the first front-view image data in a set of preset scenes. When the preset scene detected from the first front-view image data includes a specified scene, a first collection request is sent to the collection vehicle through the browser, wherein the first collection request is used to request the collection vehicle to collect scene data and save the collected scene data. This can solve the problems of high labor cost and low data collection efficiency in the scene data collection method in the related art, thereby improving the collection efficiency and reducing the labor cost.

[0099] In an exemplary embodiment, the first acquisition unit includes: a sending module, which is used to send a play request to the first server through a browser in response to a detected play trigger operation, wherein the play request is used to request to play the image data of the front-view camera; a receiving module, which is used to receive the image data of the front-view camera returned by the first server in response to the play request, and obtain first front-view image data, wherein the image data of the front-view camera is obtained by the first server by consuming the image data of the front-view camera in the second server, and the second server is a Kafka server.

[0100] The above-mentioned device also includes: a playback unit, which is used to perform scene detection on the first front view image data using a target detection model running on a browser, and after obtaining a first detection result, play the target image data on the browser, wherein the target image data is obtained by adding a scene identifier for a specified scene on the first front view image data.

[0101] In an exemplary embodiment, the first detection unit includes: a first execution module, used to input the first front view image data into the target detection model, and use a network assembly accelerator to perform model reasoning of the target detection model to obtain a first detection result, wherein the network assembly accelerator is used to utilize the browser's multi-threading and single instruction multiple data stream instruction set to accelerate the calculation.

[0102] In an exemplary embodiment, a set of preset scenes includes at least one of the following: traffic participants, traffic sign facilities; the specified scene is a preset scene selected from a set of preset scenes. The above-mentioned device also includes: a traversal unit, which is used to perform scene detection on the first front view image data using the target detection model running on the browser, and after obtaining the first detection result, traverse the first detection result until the traversal end condition is met, wherein the traversal end condition includes: traversing to the specified scene; and the first detection result has been traversed.

[0103] In an exemplary embodiment, the image data of the front-view camera is continuously acquired, and the acquisition time of each scene data acquisition by the acquisition vehicle is a specified time. The above-mentioned device also includes: a second acquisition unit, which is used to acquire the image data of the front-view camera of the acquisition vehicle through the browser to obtain the first front-view image data, and then acquire the image data of the front-view camera of the acquisition vehicle through the browser to obtain the second front-view image data; a second detection unit, which is used to perform scene detection on the second front-view image data using the target detection model to obtain a second detection result, wherein the second detection result is used to indicate a preset scene detected from the second front-view image data in a set of preset scenes; a second sending unit, which is used to send a second acquisition request to the acquisition vehicle through the browser when the preset scene detected from the second front-view image data includes a specified scene and the interval time between the current moment and the sending moment of the first acquisition request meets the interval time condition, wherein the second acquisition request is used to request the acquisition vehicle to collect scene data and save the collected scene data; wherein the interval time condition is that the interval time between the sending moment of the previous acquisition request is greater than or equal to the specified interval time, and the specified interval time is less than or equal to the specified time.

[0104] In an exemplary embodiment, the above-mentioned device also includes: a third acquisition unit, which is used to continuously acquire the coordinate information of the collection vehicle through a browser to obtain a set of coordinate information of the collection vehicle; a determination unit, which is used to determine the actual driving trajectory of the collection vehicle based on the set of coordinate information of the collection vehicle; a prompt unit, which is used to issue a deviation prompt information through a browser when the trajectory deviation between the actual driving trajectory and the expected driving trajectory is greater than or equal to a preset deviation threshold, so as to prompt the collection vehicle to deviate from the expected driving trajectory, wherein the expected driving trajectory is a driving trajectory specified for the collection vehicle based on a preset starting position point and an end position point.

[0105] In an exemplary embodiment, the above-mentioned device also includes: a processing unit, which is used to simplify the coordinate information set of the collected vehicle according to preset simplification parameters to obtain simplified coordinate information of the collected vehicle; and a display unit, which is used to display the vehicle position and driving trajectory of the collected vehicle in real time on the browser based on the simplified coordinate information of the collected vehicle.

[0106] It should be noted that the above modules can be implemented by software or hardware. For the latter, it can be implemented in the following ways, but not limited to: the above modules are all located in the same processor; or the above modules are located in different processors in any combination.

[0107] According to another aspect of the embodiments of the present application, a computer-readable storage medium is provided. The computer-readable storage medium includes a stored program, wherein the program executes the steps of any of the above method embodiments when it is run.

[0108] In an exemplary embodiment, the computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as a USB flash drive, a ROM, a RAM, a mobile hard disk, a magnetic disk, or an optical disk.

[0109] According to another aspect of the embodiments of the present application, an electronic device is also provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor is configured to execute the steps in any of the above method embodiments through the computer program. In an exemplary embodiment, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0110] For specific examples in this embodiment, reference may be made to the examples described in the above embodiments and exemplary implementation modes, and this embodiment will not be described in detail herein.

[0111] According to another aspect of the embodiment of the present application, a computer program product is also provided, which includes a computer program / instruction, and the computer program / instruction contains a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 709, and / or installed from the removable medium 711. When the computer program is executed by the central processing unit 701, various functions provided by the embodiment of the present application are executed. The above-mentioned serial numbers of the embodiments of the present application are only for description and do not represent the advantages and disadvantages of the embodiments.

[0112] Figure 7 The computer system structure block diagram of the electronic device used to implement the embodiment of the present application is schematically shown. Figure 7 As shown, the computer system 700 includes a CPU (Central Processing Unit) 701, which can perform various appropriate actions and processes according to the program stored in the ROM 702 or the program loaded from the storage part 708 to the RAM 703. Various programs and data required for system operation are also stored in the random access memory 703. The central processing unit 701, the read-only memory 702 and the random access memory 703 are connected to each other through a bus 704. An I / O (Input / Output) interface 705 is also connected to the bus 704.

[0113] The following components are connected to the I / O interface 705: an input section 706 including a keyboard, a mouse, etc.; an output section 707 including a CRT (Cathode Ray Tube), an LCD (Liquid Crystal Display), etc., and a speaker, etc.; a storage section 708 including a hard disk, etc.; and a communication section 709 including a network interface card such as a LAN card, a modem, etc. The communication section 709 performs communication processing via a network such as the Internet. A drive 710 is also connected to the input / output interface 705 as needed. A removable medium 711, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 710 as needed so that a computer program read therefrom is installed into the storage section 708 as needed.

[0114] In particular, according to an embodiment of the present application, the process described in each method flow chart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a computer readable medium, and the computer program contains a program code for executing the method shown in the flow chart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 709, and / or installed from the removable medium 711. When the computer program is executed by the central processor 701, various functions defined in the system of the present application are executed.

[0115] It should be noted that Figure 7 The computer system 700 of the electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0116] Obviously, those skilled in the art should understand that the above modules or steps of the present application can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order from that herein, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present application is not limited to any specific combination of hardware and software.

[0117] The above are only preferred embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the principles of the present application shall be included in the protection scope of the present application.

Claims

1. A method for collecting scene data, characterized in that: include: Acquire the image data of the front-view camera of the vehicle through the browser to obtain the first front-view image data; Using the target detection model run by the browser to perform scene detection on the first front view image data, to obtain a first detection result, wherein the target detection model is a model for detecting a set of preset scenes, and the first detection result is used to indicate a preset scene detected from the first front view image data in the set of preset scenes; In the case where the preset scene detected from the first forward-view image data includes a designated scene, a first acquisition request is sent to the acquisition vehicle through the browser, wherein the first acquisition request is used to request the acquisition vehicle to acquire scene data and save the acquired scene data.

2. The method according to claim 1, characterized in that The method of acquiring the image data of the front-view camera of the vehicle through the browser to obtain the first front-view image data includes: in response to the detected play trigger operation, sending a play request to the first server through the browser, wherein the play request is used to request to play the image data of the front-view camera; receiving the image data of the front-view camera returned by the first server in response to the play request to obtain the first front-view image data, wherein the image data of the front-view camera is obtained by the first server by consuming the image data of the front-view camera in the second server, and the second server is a Kafka server; After performing scene detection on the first front view image data using the target detection model running on the browser to obtain a first detection result, the method further includes: playing the target image data on the browser, wherein the target image data is obtained by adding a scene identifier for the designated scene on the first front view image data.

3. The method according to claim 1, characterized in that The using the target detection model running on the browser to perform scene detection on the first front-view image data to obtain a first detection result includes: The first front view image data is input into the target detection model, and a network assembly accelerator is used to perform model reasoning of the target detection model to obtain the first detection result, wherein the network assembly accelerator is used to utilize the multi-threading and single instruction multiple data stream instruction set of the browser to accelerate the calculation.

4. The method according to claim 1, characterized in that: The set of preset scenes includes at least one of the following: traffic participants, traffic sign facilities; the specified scene is a preset scene selected from the set of preset scenes; After performing scene detection on the first front-view image data using the target detection model running on the browser to obtain a first detection result, the method further includes: The first detection result is traversed until a traversal end condition is satisfied, wherein the traversal end condition includes: traversing to the specified scene; and the traversal of the first detection result is completed.

5. The method according to claim 1, characterized in that The image data of the front-view camera is continuously acquired, and the acquisition time of each scene data acquisition by the acquisition vehicle is a specified time; After acquiring the image data of the front-view camera of the vehicle through the browser to obtain the first front-view image data, the method further includes: Acquire the image data of the front-view camera of the vehicle through the browser to obtain second front-view image data; Using the target detection model to perform scene detection on the second front view image data to obtain a second detection result, wherein the second detection result is used to indicate a preset scene detected from the second front view image data in the set of preset scenes; When the preset scene detected from the second forward-view image data includes the specified scene, and the interval between the current time and the time when the first acquisition request is sent meets the interval time condition, sending a second acquisition request to the acquisition vehicle through the browser, wherein the second acquisition request is used to request the acquisition vehicle to collect scene data and save the collected scene data; The interval time condition is that the interval time between the sending time of the previous acquisition request is greater than or equal to the specified interval time, and the specified interval time is less than or equal to the specified time.

6. The method according to claim 1, characterized in that The method further comprises: Continuously acquiring the coordinate information of the collected vehicle through the browser to obtain a set of coordinate information of the collected vehicle; Determining the actual driving trajectory of the collected vehicle based on the coordinate information set of the collected vehicle; When the trajectory deviation between the actual driving trajectory and the expected driving trajectory is greater than or equal to a preset deviation threshold, a deviation prompt message is issued through the browser to prompt the collection vehicle to deviate from the expected driving trajectory, wherein the expected driving trajectory is a driving trajectory specified for the collection vehicle based on a preset starting position point and an end position point.

7. The method according to claim 6, characterized in that The method further comprises: Performing simplification tolerance processing on the coordinate information set of the collected vehicle according to preset simplification parameters to obtain simplified coordinate information of the collected vehicle; The vehicle position of the collected vehicle and the driving track of the collected vehicle are displayed in real time on the browser based on the simplified coordinate information of the collected vehicle.

8. A scene data collection device, characterized in that: include: A first acquisition unit is used to acquire image data of a front-view camera of a vehicle through a browser to obtain first front-view image data; A first detection unit performs scene detection on the first front view image data using a target detection model running on the browser to obtain a first detection result, wherein the target detection model is a model for detecting a set of preset scenes, and the first detection result is used to indicate a preset scene detected from the first front view image data in the set of preset scenes; The first sending unit is used to send a first acquisition request to the acquisition vehicle through the browser when the preset scene detected from the first forward-view image data includes a specified scene, wherein the first acquisition request is used to request the acquisition vehicle to collect scene data and save the collected scene data.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program implements the steps of any one of the methods of claims 1 to 7 when executed by a processor.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

Images