Vehicle detection method, device, system and equipment and storage medium

By receiving and processing media stream data transmitted by the client and using vehicle detection models for detection, the problems of low vehicle detection efficiency and low accuracy in the prior art are solved, automated detection is realized, and equipment performance requirements are reduced.

CN119068165APending Publication Date: 2024-12-03BEIJING DONGCHEZU TECHNOLOGY CO LTD
View PDF 4 Cites 0 Cited by

Patent Information

Application Number
CN202310632312.3
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2023-05-30
Publication Date
2024-12-03

AI Technical Summary

Technical Problem

In the prior art, vehicle inspection mainly relies on manual observation and manual entry, and lacks automated solutions, resulting in low efficiency and low detection accuracy.

Method used

By receiving the media stream data transmitted by the client, inputting it into the vehicle detection model for detection, generating detection results, and transmitting the results back to the client to realize automated detection of the vehicle.

Benefits of technology

It realizes automated vehicle detection, reduces the requirements for client equipment performance, and improves detection efficiency and accuracy.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119068165A_ABST
    Figure CN119068165A_ABST
Patent Text Reader

Abstract

The invention relates to a vehicle detection method, device, system and equipment and a storage medium. The method comprises the following steps: receiving media stream data transmitted by a client; inputting a first image frame in the media stream data into a vehicle detection model, and detecting the first image frame by using the vehicle detection model to obtain a vehicle detection result corresponding to the first image frame; and transmitting the vehicle detection result to the client. According to the embodiment of the invention, the media data stream is transmitted to the server side, the server side is used for processing the media data stream, the vehicle detection result is obtained, automatic detection of the vehicle is realized, and in addition, the requirement of vehicle detection on the performance of client equipment is reduced.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present disclosure relates to the field of computer processing technology, and in particular to a vehicle detection method, device, equipment and storage medium. Background Art

[0002] Currently, vehicle inspections are mostly performed by inspectors observing the vehicle body with the naked eye. When defects are found, they are manually entered into the equipment and screenshots are taken.

[0003] With the development of the Internet industry, the scale of artificial intelligence applications continues to expand. Artificial Intelligence (AI) can be used to assist inspectors in testing and realize automated vehicle testing. However, how to achieve automated vehicle testing is a technical problem that needs to be solved urgently. Summary of the invention

[0004] In order to solve the above technical problems, the embodiments of the present disclosure provide a vehicle detection method, apparatus, device and storage medium, which transmit media streaming data to a server for processing, realize automatic vehicle detection, and reduce the requirements of vehicle detection on client device performance.

[0005] In a first aspect, an embodiment of the present disclosure provides a vehicle detection method, including:

[0006] Receive media stream data transmitted by the client;

[0007] Inputting the first image frame in the media stream data into a vehicle detection model, detecting the first image frame using the vehicle detection model, and obtaining a vehicle detection result corresponding to the first image frame; wherein the vehicle detection result is used to reflect the state of the target vehicle in the media stream data;

[0008] The vehicle detection result is transmitted to the client.

[0009] In an optional implementation manner, before the receiving client transmits the media stream data, the method further includes:

[0010] Establishing a real-time communication connection with the client;

[0011] Correspondingly, the receiving of media stream data transmitted by the client includes:

[0012] Based on the real-time communication connection, media stream data transmitted by the client is received.

[0013] In an optional implementation manner, the method further includes:

[0014] After receiving a video acquisition request from the client, the target video corresponding to the media stream data is sent to the client; wherein the target video is obtained by converting the media stream data.

[0015] In an optional implementation manner, the method further includes:

[0016] The vehicle detection result includes an orientation matching value corresponding to the first image frame and vehicle orientation information, wherein the orientation matching value is used to characterize the accuracy of the orientation reflected by the first image. The method further includes:

[0017] If it is determined that the orientation matching value of the first image frame reaches the orientation matching threshold corresponding to the vehicle orientation information, performing format conversion on the first image frame to obtain a first picture;

[0018] Accordingly, transmitting the vehicle detection result to the client includes:

[0019] The corresponding position matching value, the vehicle position information and the first picture are transmitted to the client.

[0020] In an optional implementation manner, the method further includes:

[0021] If it is determined that the orientation matching value corresponding to the first image frame reaches the orientation matching threshold corresponding to the vehicle orientation information, the orientation matching value corresponding to the first image frame is used to update the orientation matching threshold corresponding to the vehicle orientation information.

[0022] In an optional implementation manner, the method further includes:

[0023] The vehicle detection result includes vehicle defect information, and the vehicle defect information is used to display the vehicle defects represented by the vehicle defect information on the collection page of the media stream data of the client.

[0024] In an optional implementation manner, before transmitting the vehicle detection result to the client, the method further includes:

[0025] Convert the first image frame into a second image;

[0026] Accordingly, transmitting the vehicle detection result to the client includes:

[0027] The vehicle defect information and the second image having a corresponding relationship are transmitted to the client.

[0028] In an optional implementation manner, the method further includes:

[0029] The vehicle defect information includes a defective vehicle part identification, defect location information and a defect category identification. The defective vehicle part identification is used to identify the vehicle part where the vehicle defect reflected by the vehicle inspection result exists. The defect category identification is used to identify the defect category to which the vehicle defect reflected by the vehicle inspection result belongs. The defect location information is used to characterize the location information of the vehicle defect on the first image frame.

[0030] In a second aspect, an embodiment of the present disclosure provides a vehicle detection method, including:

[0031] Collect media stream data;

[0032] The media stream data is transmitted to a server, wherein the server is used to detect a first image frame in the media stream data using a vehicle detection model to obtain a vehicle detection result corresponding to the first image frame;

[0033] The vehicle detection result is received.

[0034] In an optional implementation manner, after receiving the vehicle detection result, the method further includes:

[0035] If it is determined that the vehicle detection result includes vehicle position information, a position identifier corresponding to the vehicle position information is displayed on the collection page of the media stream data.

[0036] In an optional implementation manner, the method further includes:

[0037] Receive a first picture; wherein the first picture is obtained by converting the format of the first image frame;

[0038] The first picture is displayed based on the position identifier corresponding to the vehicle position information.

[0039] In an optional implementation, the vehicle detection result further includes a position matching value corresponding to the first image frame, and the position identifier corresponding to the vehicle position information is based on the position identifier, and before displaying the first image, the method further includes:

[0040] Comparing the orientation matching value corresponding to the first image frame with the orientation matching value of the currently displayed picture corresponding to the vehicle orientation information;

[0041] Accordingly, displaying the first picture based on the position identifier corresponding to the vehicle position information includes:

[0042] If it is determined that the orientation matching value of the first image frame is greater than the orientation matching value of the currently displayed picture corresponding to the vehicle orientation information, the currently displayed picture corresponding to the vehicle orientation information is updated using the first picture based on the orientation identifier corresponding to the vehicle orientation information.

[0043] In an optional implementation manner, after receiving the vehicle detection result, the method further includes:

[0044] If it is determined that the vehicle detection result includes vehicle defect information, the vehicle defect corresponding to the vehicle defect information is displayed on the collection page of the media stream data.

[0045] In an optional implementation manner, the vehicle defect information includes defect location information, defective vehicle component identification, and defect category identification. If it is determined that the vehicle inspection result includes vehicle defect information, the vehicle defect corresponding to the vehicle defect information is displayed on the collection page of the media stream data. The method further includes:

[0046] If it is determined that the vehicle inspection result includes vehicle defect information, then on the collection page of the media stream data, based on the defect location information, the vehicle component corresponding to the defective vehicle component identifier is marked and the defect category corresponding to the defect category identifier is displayed.

[0047] In an optional implementation manner, the method further includes:

[0048] Receiving a second picture; wherein the second picture is obtained by converting the format of the first image frame;

[0049] Based on the defect location information, the vehicle component corresponding to the defective vehicle component identifier is marked on the second image, and the defect category corresponding to the defect category identifier is displayed.

[0050] In an optional implementation manner, the method further includes:

[0051] Based on the defective vehicle component identifier and the defect category identifier, multiple pieces of vehicle defect information are deduplicated.

[0052] In a third aspect, an embodiment of the present disclosure provides a vehicle detection device, including:

[0053] A first receiving module, used for receiving media stream data transmitted by a client;

[0054] A detection module, used to input the first image frame in the media stream data into a vehicle detection model, and use the vehicle detection model to detect the first image frame to obtain a vehicle detection result corresponding to the first image frame; wherein the vehicle detection result is used to reflect the state of the target vehicle in the media stream data;

[0055] The first transmission module is used to transmit the vehicle detection result to the client.

[0056] In a fourth aspect, an embodiment of the present disclosure provides a vehicle detection device, including:

[0057] A collection module, used for collecting media stream data;

[0058] A second transmission module is used to transmit the media stream data to a server; wherein the server is used to detect the first image frame in the media stream data using a vehicle detection model to obtain a vehicle detection result corresponding to the first image frame;

[0059] The second receiving module is used to receive the vehicle detection result.

[0060] In a fifth aspect, an embodiment of the present disclosure provides a vehicle detection system, the system comprising: a client and a server, wherein the server is used to execute the vehicle detection method as described in any one of the first aspect above, and the client is used to execute the vehicle detection method as described in any one of the second aspect above.

[0061] In a sixth aspect, an embodiment of the present disclosure provides an electronic device, the electronic device comprising:

[0062] one or more processors;

[0063] A storage device for storing one or more programs;

[0064] When the one or more programs are executed by the one or more processors, the one or more processors implement the vehicle detection method as described in any one of the first aspects or the vehicle detection method as described in any one of the second aspects.

[0065] In the seventh aspect, an embodiment of the present disclosure provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the vehicle detection method as described in any one of the first aspect or the vehicle detection method as described in any one of the second aspect.

[0066] In an eighth aspect, an embodiment of the present disclosure provides a computer program product, which includes a computer program or instructions, and when the computer program or instructions are executed by a processor, it implements the vehicle detection method as described in any one of the first aspect or the vehicle detection method as described in any one of the second aspect.

[0067] The disclosed embodiments provide a vehicle detection method, apparatus, system, device and storage medium, the method comprising: receiving media stream data transmitted by a client; inputting the first image frame in the media stream data into a vehicle detection model, detecting the first image frame using the vehicle detection model, and obtaining a vehicle detection result corresponding to the first image frame; and transmitting the vehicle detection result to the client. The disclosed embodiments achieve automatic vehicle detection by transmitting the media data stream to a server, processing the media data stream using a vehicle detection model deployed on the server, and obtaining a vehicle detection result. In addition, the disclosed embodiments reduce the requirements of vehicle detection on the performance of client devices. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and the originals and elements are not necessarily drawn to scale.

[0069] Figure 1 An architectural diagram of a vehicle detection scenario provided in an embodiment of the present disclosure;

[0070] Figure 2 A schematic diagram of a vehicle detection method in an embodiment of the present disclosure;

[0071] Figure 3 A schematic diagram of a vehicle detection method in an embodiment of the present disclosure;

[0072] Figure 4 is an interactive schematic diagram of a vehicle detection system in an embodiment of the present disclosure;

[0073] Figure 5 is an interactive schematic diagram of another vehicle detection system in an embodiment of the present disclosure;

[0074] Figure 6 is a structural schematic diagram of a vehicle detection device in an embodiment of the present disclosure;

[0075] Figure 7 is a structural schematic diagram of a vehicle detection device in an embodiment of the present disclosure;

[0076] Figure 8It is a structural schematic diagram of an electronic device in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0077] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments described herein, which are instead provided for a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0078] It should be understood that the various steps described in the method embodiments of the present disclosure may be performed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this respect.

[0079] The term "including" and its variations used herein are open inclusions, i.e., "including but not limited to". The term "based on" means "based at least in part on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". The relevant definitions of other terms will be given in the following description.

[0080] It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0081] It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0082] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0083] Before further describing the embodiments of the present disclosure in detail, the nouns and terms involved in the embodiments of the present disclosure are described. The nouns and terms involved in the embodiments of the present disclosure are subject to the following interpretations.

[0084] In response, it is used to indicate the conditions or states on which the executed operations depend. When the dependent conditions or states are met, one or more operations executed can be in real time or have a set delay. Unless otherwise specified, there is no restriction on the order of execution of the multiple operations executed.

[0085] It is understandable that before using the technical solutions disclosed in the embodiments of the present disclosure, the types, scope of use, usage scenarios, etc. of the personal information involved in the present disclosure should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.

[0086] For example, in response to receiving an active request from a user, a prompt message is sent to the user to clearly prompt the user that the operation requested to be performed will require obtaining and using the user's personal information. Thus, the user can autonomously choose whether to provide personal information to software or hardware such as an electronic device, application, server, or storage medium that performs the operation of the technical solution of the present disclosure according to the prompt message.

[0087] The vehicle detection method provided in the embodiments of the present disclosure is described in detail below in conjunction with the accompanying drawings.

[0088] Figure 1 A system for implementing vehicle detection provided by an embodiment of the present disclosure is provided. Figure 1 As shown, the system 100 may include a plurality of user terminals 110, a network 120, a server 130, and a database 140. For example, the user terminal 110 may be used to implement the vehicle detection method described in any embodiment of the present disclosure.

[0089] It will be appreciated that the user terminal 110 may be any other type of electronic device capable of performing data processing, which may include but is not limited to: a mobile phone, a station, a unit, a device, a multimedia computer, a multimedia tablet, an Internet node, a communicator, a desktop computer, a laptop computer, a notebook computer, a netbook computer, a tablet computer, a personal communication system (PCS) device, a personal navigation device, a personal digital assistant (PDA), an audio / video player, a digital camera / camcorder, a positioning device, a television receiver, a radio broadcast receiver, an e-book device, a gaming device, or any combination thereof, including accessories and peripherals of these devices or any combination thereof.

[0090] The embodiments of the present disclosure have no restrictions on the hardware system and software system of the user terminal 110. For example, the user terminal 110 can be based on a processor such as ARM, X86, etc., can have input / output devices such as a camera, a touch screen, and a microphone, and can run an operating system such as Windows, iOS, Linux, Android, and Hongmeng OS.

[0091] The user terminal 110 can implement the vehicle detection method provided by the embodiment of the present disclosure by running a process or thread. In some examples, the user terminal 110 can use its built-in application to execute the vehicle detection method. In other examples, the user terminal 110 can execute the vehicle detection method by calling an application stored externally to the user terminal 110.

[0092] The network 120 may be a single network or a combination of at least two different networks. For example, the network 120 may include, but is not limited to, a local area network, a wide area network, a public network, a private network, etc., or a combination of several of them. The network 120 may be a computer network such as the Internet and / or various telecommunication networks (e.g., 3G / 4G / 5G mobile communication networks, Wi-Fi, Bluetooth, ZigBee, etc.), and the embodiments of the present disclosure are not limited thereto.

[0093] The server 130 may be a single server, or a server group, or a cloud server, and each server in the server group is connected via a wired or wireless network. A server group may be centralized, such as a data center, or distributed. The server 130 may be local or remote. The server 130 may communicate with the user terminal 110 via a wired or wireless network. The embodiments of the present disclosure do not limit the hardware system and software system of the server 130.

[0094] The database 140 may generally refer to a device with a storage function. The database 140 is mainly used to store various data used, generated and output by the user terminal 110 and the server 130 during work. The database 140 may be local or remote. The database 140 may include various memories, such as random access memory (RAM), read-only memory (ROM), etc. The storage devices mentioned above are just some examples, and the storage devices that can be used by the system 100 are not limited thereto. The embodiments of the present disclosure do not limit the hardware system and software system of the database 140. For example, it may be a relational database or a non-relational database.

[0095] The database 140 may be connected or communicated with the server 130 or a part thereof via the network 120 , or directly connected or communicated with the server 130 , or a combination of the above two methods.

[0096] In some examples, the database 140 may be an independent device. In other examples, the database 140 may also be integrated in at least one of the user terminal 110 and the server 130. For example, the database 140 may be set on the user terminal 110 or on the server 130. For another example, the database 140 may also be distributed, with a portion of it being set on the user terminal 110 and another portion being set on the server 130.

[0097] Figure 2 is a flow chart of a vehicle detection method in an embodiment of the present disclosure. This embodiment is applicable to the case of online evaluation of vehicles. The method can be executed by a vehicle detection device. The vehicle detection device can be implemented in software and / or hardware. The vehicle detection device can be configured as follows: Figure 1 in the server.

[0098] like Figure 2 As shown, the vehicle detection method provided by the embodiment of the present disclosure mainly includes steps S101-S103.

[0099] S101: Receive media stream data transmitted by a client.

[0100] In the embodiment of the present disclosure, the media stream data can be understood as video data transmitted in the form of a stream. Figure 1 The electronic device described in the embodiment of the present invention) travels around the vehicle for one week, and the media stream data generated in the process of traveling around the vehicle is transmitted to the server through the client. The service point can be a node server.

[0101] In an embodiment of the present disclosure, before receiving the media stream data transmitted by the client, the method includes: establishing a real-time communication connection with the client, specifically, establishing a real-time communication connection with the client based on the WebRTC communication protocol; receiving the media stream data transmitted by the client, including: receiving the media stream data transmitted by the client based on the real-time communication connection established above.

[0102] In one embodiment of the present disclosure, the server obtains the camera information sent by the client through the socket protocol, and the server creates a server WebRTC based on the camera information, registers the service information, monitors the media stream information, and after successful creation, returns the interactive connection establishment (ICE) service Servers information (also known as iceServers information) to the client, and the client creates a client WebRTC based on the iceServers information. The client WebRTC and the server WebRTC communicate based on the turn network protocol or the stun network protocol to establish a real-time communication connection based on the WebRTC communication protocol. The camera information includes the camera resolution.

[0103] In one embodiment of the present disclosure, media stream data is transmitted via a WebRTC real-time communication connection to achieve real-time transmission of the stream data.

[0104] S102. Input the first image frame in the media stream data into a vehicle detection model, and use the vehicle detection model to detect the first image frame to obtain a vehicle detection result corresponding to the first image frame; wherein the vehicle detection result is used to reflect the state of the target vehicle in the media stream data.

[0105] In one embodiment of the present disclosure, the vehicle detection model includes a vehicle orientation recognition model and a vehicle defect detection model. The vehicle orientation recognition model is used to identify the vehicle orientation corresponding to the target vehicle in the image frame, and the vehicle defect detection model is used to detect the defect position in the image frame.

[0106] In the disclosed embodiment, nine standard directions are set, namely the left front, left side, left rear, rear, right rear, right side, right front, front and roof of the vehicle.

[0107] In the embodiment of the present disclosure, the vehicle detection model is deployed on an AI server. The AI ​​server and the above-mentioned node server can be deployed in the same server device or in multiple server devices, which is not specifically limited in the embodiment of the present disclosure.

[0108] In the disclosed embodiment, media stream data refers to a series of image frames arranged in a set order. In the disclosed embodiment, media stream data is sequentially input into the vehicle detection model in the order of the image frames, so as to detect the image frames using the vehicle detection model. It should be noted that the media stream data does not need to be input into the vehicle detection model after the acquisition is completed. Instead, after the client's camera acquires the image frame, the image frame is transmitted to the node server in real time, and the node server transmits the image frame to the vehicle detection model for detection in real time, so as to realize real-time detection of the vehicle.

[0109] In one embodiment of the present disclosure, the first image frame in the media stream data is input into a vehicle detection model, and the first image frame is detected by using the vehicle detection model to obtain a vehicle detection result corresponding to the first image frame, including: performing frame extraction processing on the media stream data to obtain a first image frame; inputting the first image frame into a vehicle detection model, so that the vehicle detection model detects the first image frame, and obtains a vehicle detection result corresponding to the first image frame, wherein the vehicle detection result is used to reflect the status of the target vehicle in the media stream data.

[0110] In one embodiment of the present disclosure, since a series of image frames included in the media stream data have a large overlap of image content in adjacent image frames, a first image frame can be extracted from the video stream data by frame extraction and input into the vehicle detection model. In other words, an image frame can be extracted as the first image frame after every preset number of image frames and input into the vehicle detection model. The preset number can be set according to actual conditions, for example, the preset number is 3. That is, an image frame is obtained as the first image frame every 3 frames.

[0111] In the disclosed embodiment, a first image frame is determined by frame extraction and input into a vehicle detection model to reduce the workload of the vehicle detection model.

[0112] S103: Transmit the vehicle detection result to the client.

[0113] In one embodiment of the present disclosure, after receiving the vehicle detection result corresponding to the first image frame, the vehicle detection result can be directly transmitted to the client. Alternatively, after performing some setting processing on the vehicle detection result, the processed vehicle detection result can be transmitted to the client. In an embodiment of the present disclosure, the vehicle detection result is transmitted to the client via the WebRTC communication protocol.

[0114] In one embodiment of the present disclosure, the vehicle detection result includes an orientation matching value corresponding to the first image frame and vehicle orientation information, and the orientation matching value is used to characterize the accuracy of the orientation reflected by the first image frame; when the orientation matching value corresponding to the first image frame is greater than or equal to the orientation matching threshold corresponding to the vehicle orientation information, the vehicle orientation information is transmitted to the client, so that the orientation mark corresponding to the vehicle orientation information in the client is displayed in a first manner.

[0115] In the disclosed embodiment, the vehicle orientation information includes 9 set standard orientations as described in the above embodiment. Among them, the orientation matching value corresponding to the first image frame can be used to characterize the similarity between the orientation standard images corresponding to the orientation. For example: the vehicle orientation information corresponding to the first image frame is the front orientation. The orientation matching value corresponding to the first image frame refers to the similarity between the first image frame and the orientation standard image corresponding to the front orientation. The higher the above similarity, the higher the orientation matching value, and the lower the similarity, the lower the orientation matching value.

[0116] In the embodiment of the present disclosure, the orientation matching threshold can be set according to actual conditions. In the implementation of the present disclosure, a percentage score is used, and the orientation matching threshold is 90 as an example for explanation. If the orientation matching value corresponding to the first image frame is greater than or equal to 90 points, it indicates that the vehicle orientation information corresponding to the first image frame is valid, and the vehicle orientation information is transmitted to the client, so that the orientation mark corresponding to the vehicle orientation information in the client is displayed in a first manner. The first manner may include displaying the orientation mark corresponding to the vehicle orientation information with a mark, such as lighting up the display.

[0117] In the disclosed embodiment, if the orientation matching value corresponding to the first image frame is less than 90 points, it indicates that the vehicle orientation information corresponding to the first image frame is invalid and does not need to be transmitted back to the client, but can be directly discarded on the server side.

[0118] In the embodiment of the present disclosure, each standard vehicle orientation corresponds to a orientation mark. When the server has not transmitted back the vehicle orientation information, the orientation mark corresponding to the vehicle orientation information is displayed in gray. When the server transmits back the vehicle orientation information, the orientation mark corresponding to the vehicle orientation information is lit and displayed, thereby prominently prompting the inspector of the specific orientation that has been successfully detected.

[0119] In one embodiment of the present disclosure, the vehicle position recognition result includes 9 standard vehicle positions, and each standard position corresponds to a vehicle position picture. If the number of standard vehicle positions obtained after the detection is less than 9, it indicates that there is an error in the detection process, and the vehicle to be detected needs to be re-detected.

[0120] In one embodiment of the present disclosure, the method also includes: processing the first image frame corresponding to the vehicle orientation information into a first picture when the orientation matching value corresponding to the image frame is greater than or equal to an orientation matching threshold; and transmitting the first picture, the vehicle orientation information corresponding to the first picture, and the orientation matching value corresponding to the first picture to the client.

[0121] In an embodiment of the present disclosure, when the orientation matching value corresponding to the first image frame is greater than or equal to the orientation matching threshold, it indicates that the vehicle orientation information corresponding to the first image frame is valid, and the first image frame corresponding to the vehicle orientation information is processed into a picture as the first picture, and the first picture, the vehicle orientation information corresponding to the first picture, and the orientation matching value corresponding to the first image piece are transmitted to the client so that the client can perform further processing.

[0122] In one embodiment of the present disclosure, the vehicle detection result includes vehicle defect information corresponding to the first image frame, and the vehicle defect information is used to display the vehicle defect represented by the vehicle defect information on the acquisition page of the media stream data of the client. Before transmitting the vehicle detection result to the client, it also includes: converting the format of the first image frame to obtain a second image, and then transmitting the corresponding vehicle defect information and the second image to the client.

[0123] In the embodiment of the present disclosure, the vehicle defect information includes a defective vehicle part identification, defect location information and a defect category identification. The defective vehicle part identification is used to identify the vehicle part having the vehicle defect reflected by the vehicle inspection result. The defect category identification is used to identify the defect category to which the vehicle defect reflected by the vehicle inspection result belongs. The defect location information is used to characterize the location information of the vehicle defect on the first image frame.

[0124] In an embodiment of the present disclosure, the defect location information may be a closed figure represented by coordinates, such as a quadrilateral, a circle, an ellipse, etc. The defect location information is transmitted to the client through the WebRTC communication protocol, so that the client displays the defect location information in a second manner. For example, the defect location information is marked on a display page of the client.

[0125] In an embodiment of the present disclosure, the media stream data collected by the camera is played in real time on the display screen of the electronic device, that is, the video screen corresponding to the media stream data is played on the display screen of the electronic device, and the defect location information is superimposed and displayed on the video screen.

[0126] In one embodiment of the present disclosure, it also includes: processing the defect location information to obtain a defective vehicle component identification corresponding to the defect location information; processing the first image frame corresponding to the defect location information into a second picture; and transmitting the defect location information, the defective vehicle component identification and the second picture to the client.

[0127] In the disclosed embodiment, by performing component area matching on the defect location information, the defective vehicle component identification corresponding to the defect location information is obtained, and the defect location information, the defective vehicle component identification and the second image are transmitted to the client via the WebRTC communication protocol.

[0128] In one embodiment of the present disclosure, the method further includes: processing the media stream data using ffmpeg processing technology to obtain a target video; after receiving a video request sent by the client, sending the target video to the client.

[0129] In one embodiment of the present disclosure, the media stream data is converted into video information using ffmpeg processing technology as the target video. The video information refers to multimedia content stored in a video format.

[0130] In one embodiment of the present disclosure, after the client detects that the inspector has completed the inspection, it sends a video acquisition request to the server, and the server sends the target video to the client, so that the client saves the inspection video information.

[0131] The disclosed embodiment provides a vehicle detection method, the method comprising: receiving media stream data transmitted by a client; inputting a first image frame in the media stream data into a vehicle detection model, detecting the first image frame using the vehicle detection model, obtaining a vehicle detection result corresponding to the first image frame; and transmitting the vehicle detection result to the client. The disclosed embodiment transmits the media data stream to a server, processes the media data stream using the server, and obtains a vehicle detection result, thereby reducing the requirements of vehicle detection on the performance of the client device.

[0132] Figure 3 is a flow chart of a vehicle detection method in an embodiment of the present disclosure. This embodiment is applicable to the case of online evaluation of vehicles. The method can be executed by a vehicle detection device. The vehicle detection device can be implemented in software and / or hardware. The vehicle detection device can be configured as follows: Figure 1 In the electronic device.

[0133] like Figure 3 As shown, the vehicle detection method provided by the embodiment of the present disclosure mainly includes steps S201-S203.

[0134] S201: Collect media stream data.

[0135] In the embodiment of the present disclosure, the media stream data can be understood as video data transmitted in the form of a stream. Figure 1 The electronic device described in the above) drives around the vehicle for one circle, and the media stream data is generated in the process of driving around the vehicle. Furthermore, the media stream data refers to the video stream data collected by the inspector holding the inspection device starting from the left front position of the vehicle to be inspected and driving around the vehicle from the left side (the order of driving around the vehicle is: left front, left side, left rear, rear, right rear, right side, right front, front and roof).

[0136] In one embodiment of the present disclosure, collecting the media stream data includes: in response to the inspector starting the inspection system, controlling the camera on the inspection device to turn on, and collecting the media stream data in real time through the camera.

[0137] In a specific embodiment of the present disclosure, in response to detecting a trigger operation of a start button, a getUserMedia function is called to obtain camera information, media stream data is collected through the getUserMedia function, and media elements in the media stream are played.

[0138] In one embodiment of the present disclosure, after calling the getUserMedia function to obtain the camera information, the camera information is sent to the server through the socket protocol, so that the server creates a server WebRTC based on the camera information. After the creation is successful, the iceServers information is returned to the client, and the client creates a client WebRTC based on the iceServers information. The client WebRTC and the server WebRTC communicate based on the turn network protocol or the stun network protocol. The camera information includes the camera resolution.

[0139] S202: Transmit the media stream data to the server, wherein the server is configured to detect a first image frame in the media stream data using a vehicle detection model to obtain a vehicle detection result corresponding to the first image frame.

[0140] In one embodiment of the present disclosure, media stream data is transmitted via a WebRTC communication link to achieve real-time transmission of the stream data.

[0141] In the embodiment of the present disclosure, the specific process of "the server is used to detect the first image frame in the media stream data using the vehicle detection model to obtain the vehicle detection result corresponding to the first image frame" refers to the description in the above embodiment and is no longer specifically limited in the embodiment of the present disclosure.

[0142] S203: Receive the vehicle detection result.

[0143] In an embodiment of the present disclosure, when the vehicle detection result includes vehicle position information, the position mark corresponding to the vehicle position information is displayed in a first manner.

[0144] In the embodiment of the present disclosure, each standard vehicle orientation corresponds to a orientation identifier. When the server has not transmitted back the vehicle orientation information, the method identifier corresponding to the vehicle orientation information is displayed in gray. When the server transmits back the vehicle orientation information, the method identifier corresponding to the vehicle orientation information is lit and displayed, thereby prominently prompting the inspector of the specific orientation that has been successfully detected.

[0145] In an embodiment of the present disclosure, when the vehicle inspection result includes defect location information, the defect location information is displayed in a second manner.

[0146] In an embodiment of the present disclosure, the media stream data collected by the camera is played in real time on the display screen of the electronic device, that is, the video screen corresponding to the media stream data is played on the display screen of the electronic device, and the defect location information is superimposed and displayed on the video screen. For example, the defect location information can be a closed figure represented by coordinates, such as a quadrilateral, a circle, an ellipse, etc., and the above closed figure is superimposed and displayed on the video screen. This allows the inspector to intuitively see which positions have defects during the inspection process.

[0147] In one embodiment of the present disclosure, the vehicle detection result includes the first picture, the vehicle orientation information corresponding to the first picture, and the orientation matching value corresponding to the first picture, and the method also includes: receiving the first picture, the vehicle orientation information corresponding to the first picture, and the orientation matching value corresponding to the first picture; comparing the orientation matching value corresponding to the first picture with the orientation matching threshold of the vehicle orientation information corresponding to the first picture; if the orientation matching value corresponding to the first picture is greater than the orientation matching threshold, updating the orientation matching threshold using the orientation matching value corresponding to the first picture, that is, determining the orientation matching value corresponding to the first picture as the orientation matching threshold corresponding to the vehicle orientation information, and the updated access matching threshold is used to compare with the orientation matching value corresponding to the vehicle access information received subsequently.

[0148] In the disclosed embodiment, a corresponding orientation picture is stored for each standard orientation. If the orientation matching value of the first picture is greater than the orientation matching threshold of the vehicle orientation information corresponding to the first picture, the first picture is stored as the orientation picture corresponding to the vehicle orientation information. If the orientation matching value of the first picture is less than the orientation matching threshold of the vehicle orientation information corresponding to the first picture, the first picture and the orientation matching value corresponding to the first picture are discarded.

[0149] In the disclosed embodiment, the above processing is performed on each picture corresponding to each vehicle orientation information, so that the orientation picture of each standard orientation is the picture with the highest orientation matching value for that orientation.

[0150] In one embodiment of the present disclosure, the vehicle detection result includes defect location information corresponding to the second image, defective vehicle component identification and defect category identification corresponding to the second image, and the second image, and the method further includes: performing deduplication processing based on the defect location information corresponding to the second image and the pre-obtained defect location information; performing deduplication processing based on the defective vehicle component identification corresponding to the second image and the pre-obtained defective vehicle component identification, and performing deduplication processing based on the defect category identification corresponding to the second image and the pre-obtained defect category identification. The embodiment of the present disclosure can perform deduplication on the vehicle defect information obtained by vehicle detection based on one or more of the above deduplication processing.

[0151] In the disclosed embodiment, the pre-obtained defect location information refers to the defect location information of the vehicle pre-stored in the client. The pre-obtained defect location information may be defect location information manually input by an inspector, or may be defect location information extracted from maintenance data or accident records.

[0152] In the disclosed embodiment, deduplication processing is performed based on the defect location information corresponding to the second image and the pre-obtained defect location information, which means that only one piece of defect location information corresponding to the same defect is retained, thereby ensuring unique storage of the defect location information.

[0153] In one embodiment of the present disclosure, in response to the inspector's triggering operation on the end inspection control, a video acquisition request is generated and sent to the server. After receiving the video acquisition request, the server transmits the target video to the client so that the client saves the target video.

[0154] In one embodiment of the present disclosure, in response to the inspector's triggering operation on the inspection enhancement control, the vehicle orientation picture corresponding to each vehicle orientation information, the vehicle defect picture (i.e., the second picture mentioned above) and the defective vehicle part identification are sent to the server, so that the server adds the above information to the pre-set inspection report template to obtain the inspection report corresponding to the vehicle.

[0155] Figure 4 FIG. 1 is an interactive diagram of a vehicle detection system in an embodiment of the present disclosure. The present embodiment is applicable to the case of online vehicle evaluation. The system may include a client and a server. Figure 4 As shown, the interaction process of the vehicle detection system provided in the embodiment of the present disclosure mainly includes the following steps:

[0156] S401 : The client 41 collects media stream data.

[0157] S402 , the client 41 transmits the media stream data to the server 42 .

[0158] S403, after receiving the media stream data transmitted by the client, the server 42 inputs the first image frame in the media stream data into the vehicle detection model, detects the first image frame using the vehicle detection model, and obtains a vehicle detection result corresponding to the first image frame.

[0159] S404 , the server 42 receives the vehicle detection result corresponding to the first image frame.

[0160] S405 , the server 42 sends the received vehicle detection result to the client 41 .

[0161] S406 , the client 41 receives the vehicle detection result transmitted by the server 42 .

[0162] The vehicle detection system provided in the embodiment of the present disclosure can execute the steps executed in the vehicle detection method provided in the method embodiment of the present disclosure, and the execution steps and beneficial effects are no longer repeated here.

[0163] Figure 5 is an interactive schematic diagram of a vehicle detection system in an embodiment of the present disclosure, such as Figure 5 As shown, the interaction process of the vehicle detection system provided in the embodiment of the present disclosure mainly includes the following steps: the client 51 responds to the triggering operation of the start button, calls the getUserMedia function to obtain the camera information, uses the camera to collect video elements and play them, and sends the camera information to the node server 52 through the socket protocol. After the node server 52 receives the camera information, it creates a server WebRTC, registers the service information, monitors the media stream information, returns the iceServers information after the registration is successful, and returns the iceServers information to the client 51 through the socket. The client 51 receives the iceServers information, creates the client WebRTC, and establishes a stun / turn service between the client WebRTC and the server WebRTC.

[0164] Then, the client 51 adds the media stream data obtained by getUserMedia to the track and sends it to the node server 52. After receiving the media stream data, the server 52 calls the orientation recognition model and defect detection model deployed by the AI ​​server 53 every 3 image frames. The AI ​​server 53 returns the vehicle orientation information and score to the node server 52. The node server 52 returns the orientation information corresponding to the image frame with a score greater than or equal to 90 points to the client 51. The client 51 records the vehicle orientation information and lights up the orientation identification icon corresponding to the vehicle orientation information. The node server 52 determines the image frame with a score greater than or equal to 90 points as valid, processes the image frame using ffmpeg technology, obtains a picture, and calls the upload service to transmit the picture and the score corresponding to the picture to the client 51. After receiving the picture and the score corresponding to the picture, the client 51 compares it with the score of the saved picture in the same orientation, and selects the picture with the highest score to save.

[0165] The AI ​​server 53 returns the defect location information to the node server 52. The node server 52 processes the defect location information to obtain the vehicle defect component identification, and returns the defect location information to the client 51. The client 51 records the defect location information and displays the defect location information on the video screen. The node server 52 determines the image frame corresponding to the defect location information as valid, processes the image frame using ffmpeg technology to obtain a picture, and calls the upload service to transmit the picture and the defect location information, the defect vehicle component identification and the defect category identification to the client 51. The client 51 receives the picture and the defect location information, the vehicle defect component identification and the defect category identification, deduplicates the defect location information, and stores each defect location information and its corresponding picture uniquely.

[0166] In response to the inspector's triggering operation on the end inspection control, the client 51 generates a video acquisition request and sends it to the node server 52. After receiving the video acquisition request, the node server 52 transmits the target video to the client 51, so that the client 51 receives and saves the target video.

[0167] Finally, in response to the inspector's triggering operation on the inspection improvement control, the client 51 sends the vehicle orientation pictures corresponding to each vehicle orientation information, as well as the vehicle defect pictures and the vehicle defect component identification to the node server 52. The node server 52 adds the above information to the pre-set inspection report template to obtain the inspection report corresponding to the vehicle.

[0168] Figure 6FIG. 1 is a schematic diagram of the structure of a vehicle detection device in an embodiment of the present disclosure. The present embodiment is applicable to the case of online evaluation of vehicles. The vehicle detection device can be implemented in software and / or hardware. The vehicle detection device can be configured as follows: Figure 1 in the server.

[0169] like Figure 6 As shown, the vehicle detection device provided by the embodiment of the present disclosure mainly includes:

[0170] The first receiving module 601 is used to receive media stream data transmitted by the client;

[0171] A detection module 602 is used to input the first image frame in the media stream data into a vehicle detection model, and use the vehicle detection model to detect the first image frame to obtain a vehicle detection result corresponding to the first image frame; wherein the vehicle detection result is used to reflect the state of the target vehicle in the media stream data;

[0172] The first transmission module 603 is used to transmit the vehicle detection result to the client.

[0173] In an optional implementation, the device further includes:

[0174] A connection establishment module, used to establish a real-time communication connection with the client;

[0175] Accordingly, the first receiving module is specifically used for:

[0176] Based on the real-time communication connection, media stream data transmitted by the client is received.

[0177] In an optional implementation, the device further includes:

[0178] The first sending module is used to send the target video corresponding to the media stream data to the client after receiving a video acquisition request from the client; wherein the target video is obtained by converting the media stream data.

[0179] In an optional implementation, the vehicle detection result includes an orientation matching value and vehicle orientation information corresponding to the first image frame, the orientation matching value is used to characterize the accuracy of the orientation reflected by the first image frame, and the device further includes:

[0180] A first conversion module, configured to convert the format of the first image frame to obtain a first picture if it is determined that the orientation matching value of the first image frame reaches the orientation matching threshold corresponding to the vehicle orientation information;

[0181] Accordingly, the first transmission module is specifically used for:

[0182] The orientation matching value, the vehicle orientation information and the first picture having a corresponding relationship are transmitted to the client.

[0183] In an optional implementation, the device further includes:

[0184] A threshold updating module is used to update the orientation matching threshold corresponding to the vehicle orientation information using the orientation matching value corresponding to the first image frame if it is determined that the orientation matching value corresponding to the first image frame reaches the orientation matching threshold corresponding to the vehicle orientation information.

[0185] In an optional implementation, the vehicle detection result includes vehicle defect information, and the vehicle defect information is used to display the vehicle defect represented by the vehicle defect information on the collection page of the media stream data of the client.

[0186] In an optional implementation, the device further includes:

[0187] A second conversion module, used for performing format conversion on the first image frame to obtain a second image;

[0188] Accordingly, the first transmission module is specifically used for:

[0189] The vehicle defect information and the second image having a corresponding relationship are transmitted to the client.

[0190] In an optional embodiment, the vehicle defect information includes a defective vehicle part identification, defect location information and a defect category identification. The defective vehicle part identification is used to identify the vehicle part having the vehicle defect reflected by the vehicle inspection result, the defect category identification is used to identify the defect category to which the vehicle defect reflected by the vehicle inspection result belongs, and the defect location information is used to characterize the location information of the vehicle defect on the first image frame.

[0191] The vehicle detection device provided in the embodiment of the present disclosure can execute the steps executed in the vehicle detection method provided in the method embodiment of the present disclosure, and the execution steps and beneficial effects are no longer repeated here.

[0192] Figure 7 FIG. 1 is a schematic diagram of the structure of a vehicle detection device in an embodiment of the present disclosure. The present embodiment is applicable to the case of online evaluation of vehicles. The vehicle detection device can be implemented in software and / or hardware. The vehicle detection device can be configured as follows: Figure 1 In the electronic device.

[0193] like Figure 7As shown, the vehicle detection device provided by the embodiment of the present disclosure mainly includes:

[0194] The acquisition module 701 is used to acquire media stream data;

[0195] The second transmission module 702 is used to transmit the media stream data to the server; wherein the server is used to detect the first image frame in the media stream data using a vehicle detection model to obtain a vehicle detection result corresponding to the first image frame;

[0196] The second receiving module 703 is used to receive the vehicle detection result.

[0197] In an optional implementation, the device further includes:

[0198] The first display module is configured to display a position mark corresponding to the vehicle position information on the collection page of the media stream data if it is determined that the vehicle detection result includes the vehicle position information.

[0199] In an optional implementation, the device further includes:

[0200] A third receiving module is used to receive a first picture; wherein the first picture is obtained by performing format conversion on the first image frame;

[0201] The second display module is used to display the first picture based on the position mark corresponding to the vehicle position information.

[0202] In an optional implementation manner, the vehicle detection result also includes an orientation matching value corresponding to the first image frame, and the device further includes:

[0203] A comparison module, configured to compare the orientation matching value corresponding to the first image frame with the orientation matching value of the currently displayed picture corresponding to the vehicle orientation information;

[0204] Accordingly, the second display module is specifically used for:

[0205] If it is determined that the orientation matching value of the first image frame is greater than the orientation matching value of the currently displayed picture corresponding to the vehicle orientation information, the currently displayed picture corresponding to the vehicle orientation information is updated using the first picture based on the orientation identifier corresponding to the vehicle orientation information.

[0206] In an optional implementation, the device further includes:

[0207] The third display module is used to display the vehicle defect corresponding to the vehicle defect information on the collection page of the media stream data if it is determined that the vehicle detection result includes vehicle defect information.

[0208] In an optional implementation manner, the vehicle defect information includes defect location information, defective vehicle component identification and defect category identification, and the third display module is specifically used to:

[0209] If it is determined that the vehicle inspection result includes vehicle defect information, then on the collection page of the media stream data, based on the defect location information, the vehicle component corresponding to the defective vehicle component identifier is marked and the defect category corresponding to the defect category identifier is displayed.

[0210] In an optional implementation, the device further includes:

[0211] A fourth receiving module, configured to receive a second picture; wherein the second picture is obtained by performing format conversion on the first image frame;

[0212] A fourth display module is used to mark the vehicle component corresponding to the defective vehicle component identifier on the second image and display the defect category corresponding to the defect category identifier based on the defect location information.

[0213] In an optional implementation, the device further includes:

[0214] A deduplication module is used to perform deduplication processing on multiple vehicle defect information based on the defective vehicle component identifier and the defect category identifier.

[0215] The vehicle detection device provided in the embodiment of the present disclosure can execute the steps executed in the vehicle detection method provided in the method embodiment of the present disclosure, and the execution steps and beneficial effects are no longer repeated here.

[0216] Figure 8 Schematic diagram of the structure of an electronic device in the embodiment of the present disclosure. Figure 8 , which shows a schematic diagram of the structure of an electronic device 800 suitable for implementing the embodiment of the present disclosure. The electronic device 800 in the embodiment of the present disclosure may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle terminals (such as vehicle navigation terminals), wearable terminal devices, etc., and fixed terminals such as digital TVs, desktop computers, smart home devices, etc. Figure 8 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0217] like Figure 8As shown, the electronic device 800 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 801, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 802 or a program loaded from a storage device 808 to a random access memory (RAM) 803 to implement the vehicle detection method of the embodiment described in the present disclosure. In RAM 803, various programs and data required for the operation of the terminal device 800 are also stored. The processing device 801, ROM 802, and RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0218] Typically, the following devices may be connected to the I / O interface 805: input devices 806 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; output devices 807 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 808 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 809. The communication devices 809 may allow the terminal device 800 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 8 The terminal device 800 is shown to have various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.

[0219] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart, thereby implementing the vehicle detection method as described above. In such an embodiment, the computer program can be downloaded and installed from the network through the communication device 809, or installed from the storage device 808, or installed from the ROM 802. When the computer program is executed by the processing device 801, the above-mentioned functions defined in the method of the embodiment of the present disclosure are executed.

[0220] It should be noted that the computer-readable medium disclosed above may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, device or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0221] In some embodiments, the client and the server may communicate using any currently known or future developed network protocol such as HTTP (HyperText Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

[0222] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0223] In a possible implementation of the present disclosure, the computer-readable medium carries one or more programs. When the one or more programs are executed by the terminal device, the terminal device: receives media stream data transmitted by the client; inputs the first image frame in the media stream data into a vehicle detection model, and uses the vehicle detection model to detect the first image frame to obtain a vehicle detection result corresponding to the first image frame; wherein the vehicle detection result is used to reflect the status of the target vehicle in the media stream data; and transmits the vehicle detection result to the client.

[0224] In a possible implementation manner of the present disclosure, when the above one or more programs are executed by the terminal device, the terminal device may also execute other steps described in the above embodiments.

[0225] The above-mentioned computer-readable medium carries one or more programs. When the above-mentioned one or more programs are executed by the terminal device, the terminal device: collects media stream data; transmits the media stream data to the server, wherein the server is used to detect the first image frame in the media stream data using a vehicle detection model to obtain a vehicle detection result corresponding to the first image frame; and receives the vehicle detection result.

[0226] In a possible implementation manner of the present disclosure, when the above one or more programs are executed by the terminal device, the terminal device may also execute other steps described in the above embodiments.

[0227] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including, but not limited to, object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0228] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0229] The units involved in the embodiments described in the present disclosure may be implemented by software or hardware, wherein the name of a unit does not, in some cases, limit the unit itself.

[0230] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0231] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0232] According to one or more embodiments of the present disclosure, the present disclosure provides a vehicle detection method, including: receiving media stream data transmitted by a client; inputting a first image frame in the media stream data into a vehicle detection model, detecting the first image frame using the vehicle detection model, and obtaining a vehicle detection result corresponding to the first image frame; and transmitting the vehicle detection result to the client.

[0233] According to one or more embodiments of the present disclosure, the present disclosure provides a vehicle detection method, including: collecting media stream data; transmitting the media stream data to the server, wherein the server is used to detect a first image frame in the media stream data using a vehicle detection model to obtain a vehicle detection result corresponding to the first image frame; and receiving the vehicle detection result.

[0234] According to one or more embodiments of the present disclosure, the present disclosure provides a vehicle detection device, including: a media stream data receiving module, used to receive media stream data transmitted by a client; a media stream data processing module, used to input a first image frame in the media stream data into a vehicle detection model, and use the vehicle detection model to detect the first image frame to obtain a vehicle detection result corresponding to the first image frame; wherein the vehicle detection result is used to reflect the status of the target vehicle in the media stream data; and a vehicle detection result sending module, used to transmit the vehicle detection result to the client.

[0235] According to one or more embodiments of the present disclosure, the present disclosure provides a vehicle detection device, including: a media stream data acquisition module, used to transmit the media stream data to a server; wherein the server is used to detect a first image frame in the media stream data using a vehicle detection model to obtain a vehicle detection result corresponding to the first image frame; a vehicle detection result receiving module, used to receive the vehicle detection result.

[0236] According to one or more embodiments of the present disclosure, the present disclosure provides a vehicle detection system, which includes: a client and a server, wherein the server is used to execute the vehicle detection method as described in any one of the first aspects above, and the client is used to execute the vehicle detection method as described in any one of the second aspects above.

[0237] According to one or more embodiments of the present disclosure, the present disclosure provides an electronic device, the electronic device comprising:

[0238] one or more processors;

[0239] A storage device for storing one or more programs;

[0240] When the one or more programs are executed by the one or more processors, the one or more processors implement the vehicle detection method as described in any one of the first aspects or the vehicle detection method as described in any one of the second aspects.

[0241] According to one or more embodiments of the present disclosure, the present disclosure provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a vehicle detection method as described in any one of the first aspects or a vehicle detection method as described in any one of the second aspects.

[0242] According to one or more embodiments of the present disclosure, the present disclosure provides a computer program product, which includes a computer program or instructions, which, when executed by a processor, implements the vehicle detection method as described in any one of the first aspects above or the vehicle detection method as described in any one of the second aspects above.

[0243] The above description is only a preferred embodiment of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosed concept. For example, the above features are replaced with the technical features with similar functions disclosed in the present disclosure (but not limited to) by each other to form a technical solution.

[0244] In addition, although each operation is described in a specific order, this should not be required to perform these operations in the specific order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details are included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.

[0245] Although the subject matter has been described in language specific to structural features and / or methodological logical actions, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. On the contrary, the specific features and actions described above are merely example forms of implementing the claims.

Claims

1. A vehicle detection method, characterized in that: The method comprises: Receive media stream data transmitted by the client; Inputting the first image frame in the media stream data into a vehicle detection model, detecting the first image frame using the vehicle detection model, and obtaining a vehicle detection result corresponding to the first image frame; wherein the vehicle detection result is used to reflect the state of the target vehicle in the media stream data; The vehicle detection result is transmitted to the client.

2. The method according to claim 1, characterized in that Before receiving the media stream data transmitted by the client, the method further includes: Establishing a real-time communication connection with the client; Correspondingly, the receiving of media stream data transmitted by the client includes: Based on the real-time communication connection, media stream data transmitted by the client is received.

3. The method according to claim 1, characterized in that The method further comprises: After receiving a video acquisition request from the client, a target video corresponding to the media stream data is sent to the client; wherein the target video is obtained by converting the media stream data.

4. The method according to claim 1, characterized in that: The vehicle detection result includes an orientation matching value and vehicle orientation information corresponding to the first image frame, wherein the orientation matching value is used to characterize the accuracy of the orientation reflected by the first image frame. The method further includes: If it is determined that the orientation matching value of the first image frame reaches the orientation matching threshold corresponding to the vehicle orientation information, performing format conversion on the first image frame to obtain a first picture; Accordingly, transmitting the vehicle detection result to the client includes: The orientation matching value, the vehicle orientation information and the first picture having a corresponding relationship are transmitted to the client.

5. The method according to claim 4, characterized in that The method further comprises: If it is determined that the orientation matching value corresponding to the first image frame reaches the orientation matching threshold corresponding to the vehicle orientation information, the orientation matching value corresponding to the first image frame is used to update the orientation matching threshold corresponding to the vehicle orientation information.

6. The method according to claim 1, characterized in that The vehicle detection result includes vehicle defect information, and the vehicle defect information is used to display the vehicle defect represented by the vehicle defect information on the collection page of the media stream data of the client.

7. The method according to claim 6, characterized in that Before transmitting the vehicle detection result to the client, the method further includes: Convert the first image frame into a second image; Accordingly, transmitting the vehicle detection result to the client includes: The vehicle defect information and the second image having a corresponding relationship are transmitted to the client.

8. The method according to claim 6 or 7, characterized in that: The vehicle defect information includes a defective vehicle part identification, defect location information and a defect category identification. The defective vehicle part identification is used to identify the vehicle part where the vehicle defect reflected by the vehicle inspection result exists. The defect category identification is used to identify the defect category to which the vehicle defect reflected by the vehicle inspection result belongs. The defect location information is used to characterize the location information of the vehicle defect on the first image frame.

9. A vehicle detection method, characterized in that: The method comprises: Collect media stream data; The media stream data is transmitted to a server; wherein the server is used to detect the first image frame in the media stream data using a vehicle detection model to obtain a vehicle detection result corresponding to the first image frame; The vehicle detection result is received.

10. The method according to claim 9, characterized in that After receiving the vehicle detection result, the method further includes: If it is determined that the vehicle detection result includes vehicle position information, a position identifier corresponding to the vehicle position information is displayed on the collection page of the media stream data.

11. The method according to claim 10, characterized in that The method further comprises: Receive a first picture; wherein the first picture is obtained by converting the format of the first image frame; The first picture is displayed based on the position identifier corresponding to the vehicle position information.

12. The method according to claim 11, characterized in that The vehicle detection result also includes a position matching value corresponding to the first image frame, and the position identifier corresponding to the vehicle position information, before displaying the first picture, further includes: Comparing the orientation matching value corresponding to the first image frame with the orientation matching value of the currently displayed picture corresponding to the vehicle orientation information; Accordingly, displaying the first picture based on the position identifier corresponding to the vehicle position information includes: If it is determined that the orientation matching value of the first image frame is greater than the orientation matching value of the currently displayed picture corresponding to the vehicle orientation information, the currently displayed picture corresponding to the vehicle orientation information is updated using the first picture based on the orientation identifier corresponding to the vehicle orientation information.

13. The method according to claim 9, characterized in that After receiving the vehicle detection result, the method further includes: If it is determined that the vehicle detection result includes vehicle defect information, the vehicle defect corresponding to the vehicle defect information is displayed on the collection page of the media stream data.

14. The method according to claim 13, characterized in that The vehicle defect information includes defect location information, defective vehicle component identification, and defect category identification. If it is determined that the vehicle inspection result includes vehicle defect information, then on the media stream data collection page, the vehicle defect corresponding to the vehicle defect information is displayed, including: If it is determined that the vehicle inspection result includes vehicle defect information, then on the collection page of the media stream data, based on the defect location information, the vehicle component corresponding to the defective vehicle component identifier is marked and the defect category corresponding to the defect category identifier is displayed.

15. The method according to claim 13 or 14, characterized in that The method further comprises: Receiving a second picture; wherein the second picture is obtained by converting the format of the first image frame; Based on the defect location information, the vehicle component corresponding to the defective vehicle component identifier is marked on the second image, and the defect category corresponding to the defect category identifier is displayed.

16. The method according to claim 14, characterized in that The method further comprises: Based on the defective vehicle component identifier and the defect category identifier, multiple pieces of vehicle defect information are deduplicated.

17. A vehicle detection device, characterized in that: The device comprises: A first receiving module, used for receiving media stream data transmitted by a client; A detection module, used to input the first image frame in the media stream data into a vehicle detection model, and use the vehicle detection model to detect the first image frame to obtain a vehicle detection result corresponding to the first image frame; wherein the vehicle detection result is used to reflect the state of the target vehicle in the media stream data; The first transmission module is used to transmit the vehicle detection result to the client.

18. A vehicle detection device, characterized in that: The device comprises: A collection module, used for collecting media stream data; A second transmission module is used to transmit the media stream data to a server; wherein the server is used to detect the first image frame in the media stream data using a vehicle detection model to obtain a vehicle detection result corresponding to the first image frame; The second receiving module is used to receive the vehicle detection result.

19. A vehicle detection system, characterized in that: The system comprises: a client and a server, wherein the server is used to execute the vehicle detection method according to any one of claims 1 to 8, and the client is used to execute the vehicle detection method according to any one of claims 9 to 16.

20. An electronic device, characterized in that: The electronic device comprises: one or more processors; A storage device for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the vehicle detection method according to any one of claims 1 to 8 or the vehicle detection method according to any one of claims 9 to 16.

21. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the vehicle detection method according to any one of claims 1 to 8 or the vehicle detection method according to any one of claims 9 to 16 is implemented.

Citation Information

Patent Citations

  • Vehicle loss assessment image obtaining method and apparatus, server and terminal device

    CN107194323A

  • Method and device for recognizing direction of vehicle body

    CN110569865A

  • A method and device for identifying category to which vehicle belongs

    CN111666898A

  • Vehicle loss assessment method and device, electronic equipment and computer readable storage medium

    CN113033372A