Vehicle question and answer method, device and equipment in technical field of intelligent cockpit and medium
By using the methods of image acquisition, depth information and direction vector calculation in the field of smart cockpit technology, the problem of large errors when determining the direction of the vehicle in front is solved, and the recognition accuracy and reliability of answers are improved.
Patent Information
- Application Number
- CN202411952907.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-27
- Publication Date
- 2025-03-28
AI Technical Summary
In the field of smart cockpit technology, the prior art has large errors in determining the direction of the vehicle in front, resulting in a higher probability of errors in the answers of the vehicle terminal to the user.
By receiving user's inquiry statements, image acquisition and detection are performed, pixel position and depth information of the target vehicle are obtained, direction vectors and azimuth angles are calculated, and the reference vectors are matched to determine whether the azimuth angle of the target vehicle matches the azimuth information of the vehicle to be identified, thereby identifying and answering.
By introducing depth information and direction vectors, the accuracy of the target vehicle's azimuth angle is improved, the recognition error is reduced, and the probability of errors occurring in the answers of the on-board terminal to the user is reduced.
Smart Images
Figure CN119723524A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of smart cockpit technology, and in particular to a vehicle question-and-answer method, device, equipment and medium in the field of smart cockpit technology. Background Art
[0002] With the development of vehicle technology, more and more vehicles have smart cockpits. Smart cockpit voice systems and gesture interactions are essential basic system applications for smart cockpits. For example, users can interact with the vehicle through the vehicle's voice assistant, including but not limited to sending instructions, checking the weather, and asking questions. In the scenario of asking questions, the user can ask the vehicle: "What is the vehicle in front of the left?" This requires the vehicle to determine which vehicle in front is the vehicle in front of the left, that is, to determine the direction of the vehicle in front.
[0003] Currently, the vehicle can obtain a front image by taking a photo of the front, then identify the front vehicle in the front image, and then determine whether the front vehicle is the vehicle to the left front of the vehicle based on the position of the front vehicle in the entire front image.
[0004] However, in the process of determining the position of the preceding vehicle, there may be a large error, which in turn increases the probability of errors in the vehicle's response to the user. Summary of the invention
[0005] The present application provides a vehicle question-and-answer method, device, equipment and medium in the field of smart cockpit technology, which can reduce the determination error and the probability of errors in the vehicle's answer to the user in the process of determining the position of the preceding vehicle.
[0006] In order to achieve the above purpose, this application adopts the following technical solutions: In a first aspect, the present application provides a vehicle question-and-answer method in the field of smart cockpit technology, the method comprising: receiving a query statement from a user, wherein the query statement includes position information of a vehicle to be identified; According to the position information of the vehicle to be identified, image acquisition is performed to obtain a target image; Detecting the target image to obtain a pixel position of a center point of a target vehicle in the target image and depth information of the target vehicle; Determine the direction vector of the target vehicle in the camera coordinate system according to the pixel position and depth information of the target vehicle and the origin position in the camera coordinate system; Calculating the orientation angle of the target vehicle according to the direction vector and the reference vector, wherein the reference vector is determined based on the origin position and the reference position; Determine whether the orientation angle of the target vehicle matches the orientation information of the vehicle to be identified, and obtain a first determination result; If the first judgment result indicates that the orientation angle of the target vehicle matches the orientation information of the vehicle to be identified, the target vehicle is identified to obtain an identification result; Based on the recognition result, a first reply statement is presented to the user.
[0007] In some possible implementations, calculating the orientation angle of the target vehicle according to the direction vector and the reference vector includes:
[0008] in, represents the azimuth angle of the target vehicle, represents the reference vector, Represents the direction vector of the i-th target vehicle.
[0009] In some possible implementations, the method further includes: Determine the magnitude relationship between the depth information of the target vehicle and a preset depth threshold to obtain a second determination result; The step of identifying the target vehicle and obtaining an identification result includes: When the second judgment result indicates that the depth information of the target vehicle is less than or equal to the preset depth threshold, the target vehicle is identified to obtain an identification result.
[0010] In some possible implementations, the method further includes: When the second judgment result indicates that the depth information of the target vehicle is greater than the preset depth threshold, a second reply statement is presented to the user, where the second reply statement is used to describe that the vehicle to be identified is not within the identification range.
[0011] In some possible implementations, the method further includes: Acquire the pixel area of the target vehicle in the target image; Determine the size relationship between the pixel area of the target vehicle and a preset area threshold to obtain a third determination result; The step of identifying the target vehicle and obtaining an identification result includes: When the third judgment result indicates that the pixel area of the target vehicle is greater than or equal to a preset area threshold, the target vehicle is identified to obtain an identification result.
[0012] In some possible implementations, determining whether the orientation angle of the target vehicle matches the orientation information of the vehicle to be identified to obtain a first determination result includes: If the azimuth angle of the target vehicle is within the angle set corresponding to the azimuth information of the vehicle to be identified, a first judgment result is obtained that the azimuth angle of the target vehicle matches the azimuth information of the vehicle to be identified; If the azimuth angle of the target vehicle does not match the azimuth information of the vehicle to be identified, a first judgment result is obtained that the azimuth angle of the target vehicle does not match the azimuth information of the vehicle to be identified.
[0013] In some possible implementations, the method further includes: If the first judgment result indicates that the orientation angle of the target vehicle does not match the orientation information of the vehicle to be identified, a third reply statement is presented to the user, wherein the third reply statement is used to describe that there is no vehicle at the orientation corresponding to the orientation information of the vehicle to be identified.
[0014] In a second aspect, the present application provides a vehicle question-and-answer device in the field of smart cockpit technology, the device comprising: An interactive module, configured to receive a user's query statement, wherein the query statement includes the position information of the vehicle to be identified; An acquisition module, used to acquire images according to the position information of the vehicle to be identified, and obtain a target image; A detection module, used to detect the target image, obtain the pixel position of the center point of the target vehicle in the target image and the depth information of the target vehicle; A vector module, used to determine the direction vector of the target vehicle in the camera coordinate system according to the pixel position and depth information of the target vehicle and the origin position in the camera coordinate system; A calculation module, used for calculating the azimuth angle of the target vehicle according to the direction vector and the reference vector, wherein the reference vector is determined based on the origin position and the reference position; A judgment module, used to judge whether the orientation angle of the target vehicle matches the orientation information of the vehicle to be identified, and obtain a first judgment result; an identification module, configured to identify the target vehicle and obtain an identification result if the first judgment result indicates that the orientation angle of the target vehicle matches the orientation information of the vehicle to be identified; The interaction module is also used to present a first reply statement to the user based on the recognition result.
[0015] In a third aspect, the present application provides a computing device, including a memory and a processor; One or more computer programs are stored in the memory, and the one or more computer programs include instructions; when the instructions are executed by the processor, the computing device executes the method as described in any one of the first aspects.
[0016] In a fourth aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium is used to store a computer program, and the computer program is used to execute the method as described in any one of the first aspects.
[0017] It can be seen from the above technical solution that the present application has at least the following beneficial effects: The present application provides a vehicle question-and-answer method in the field of intelligent cockpit technology. The method can be executed by a vehicle-mounted terminal. The method includes: the vehicle-mounted terminal receives a user's query statement, and then performs image acquisition according to the orientation information of the vehicle to be identified in the query statement to obtain a target image, and then detects the target image to obtain the pixel position of the center point of the target vehicle in the target image and the depth information of the target vehicle, and then determines the direction vector of the target vehicle in the camera coordinate system according to the pixel position and depth information of the target vehicle and the origin position in the camera coordinate system, and calculates the orientation angle of the target vehicle based on the direction vector and the reference vector. In this method, the vehicle-mounted terminal does not rely solely on single image information, but also collects depth information in the image, and further generates a three-dimensional vector (i.e., a direction vector) based on the depth information. The orientation angle of the target vehicle determined based on the direction vector introducing the depth information is more accurate, and then when the orientation angle of the target vehicle matches the orientation information of the vehicle to be identified, the target vehicle is identified to obtain an identification result, and finally, according to the identification result, the first reply statement is presented to the user. It can be seen that after the method introduces depth information, the azimuth angle determined by the direction vector generated based on the depth information is more accurate, which can reduce the recognition error and reduce the probability of errors in the vehicle-mounted terminal's answer to the user.
[0018] It should be understood that the description of technical features, technical solutions, beneficial effects or similar language in this application does not imply that all features and advantages can be realized in any single embodiment. On the contrary, it is understood that the description of features or beneficial effects means that specific technical features, technical solutions or beneficial effects are included in at least one embodiment. Therefore, the description of technical features, technical solutions or beneficial effects in this specification does not necessarily refer to the same embodiment. Furthermore, the technical features, technical solutions and beneficial effects described in the present embodiment can also be combined in any appropriate manner. Those skilled in the art will understand that the embodiment can be realized without one or more specific technical features, technical solutions or beneficial effects of a specific embodiment. In other embodiments, additional technical features and beneficial effects can also be identified in a specific embodiment that does not embody all embodiments. BRIEF DESCRIPTION OF THE DRAWINGS
[0019] Figure 1 A flow chart of a vehicle question-and-answer method in the field of smart cockpit technology provided in an embodiment of the present application; Figure 2 A schematic diagram of a camera coordinate system provided in an embodiment of the present application; Figure 3 A schematic diagram of a feature enhancement extraction module provided in an embodiment of the present application; Figure 4 A schematic diagram of a vehicle question-and-answer device in the field of smart cockpit technology provided by an embodiment of the present application; Figure 5 A schematic diagram of a computing device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0020] The terms "first", "second", "third", etc. in the specification of this application and the accompanying drawings are used to distinguish different objects rather than to limit a specific order.
[0021] In the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.
[0022] The vehicle's smart cockpit is the product of the deep integration of automotive technology and a variety of advanced technologies, providing drivers and passengers with a more intelligent and user-friendly in-car environment. Generally, the smart cockpit includes: infotainment functions, human-computer interaction functions, intelligent interconnection functions, etc. Among them, the human-computer interaction function supports multiple interaction methods such as voice recognition, gesture control, and touch control. Users can communicate with the vehicle through natural language to control various functions, such as querying information, setting navigation, adjusting equipment, etc., which greatly improves the convenience and safety of operation. During the communication between the user and the vehicle, the vehicle's answer to the user may be wrong.
[0023] The present application provides a vehicle question-and-answer method in the field of intelligent cockpit technology. The method can be executed by a vehicle-mounted terminal. The method includes: the vehicle-mounted terminal receives a user's query statement, and then performs image acquisition according to the orientation information of the vehicle to be identified in the query statement to obtain a target image, and then detects the target image to obtain the pixel position of the center point of the target vehicle in the target image and the depth information of the target vehicle, and then determines the direction vector of the target vehicle in the camera coordinate system according to the pixel position and depth information of the target vehicle and the origin position in the camera coordinate system, and calculates the orientation angle of the target vehicle based on the direction vector and the reference vector. In this method, the vehicle-mounted terminal does not rely solely on single image information, but also collects depth information in the image, and further generates a three-dimensional vector (i.e., a direction vector) based on the depth information. The orientation angle of the target vehicle determined based on the direction vector introducing the depth information is more accurate, and then when the orientation angle of the target vehicle matches the orientation information of the vehicle to be identified, the target vehicle is identified to obtain an identification result, and finally, according to the identification result, the first reply statement is presented to the user. It can be seen that after the method introduces depth information, the azimuth angle determined by the direction vector generated based on the depth information is more accurate, which can reduce the recognition error and reduce the probability of errors in the vehicle-mounted terminal's answer to the user.
[0024] In order to make the technical solution of the present application clearer and easier to understand, the following is an introduction to a vehicle question-and-answer method in the field of smart cockpit technology provided by an embodiment of the present application in conjunction with a flow chart. Figure 1 As shown, the figure is a flow chart of a vehicle question-and-answer method in the field of smart cockpit technology provided by an embodiment of the present application.
[0025] The embodiment of the present application is executed by a vehicle-mounted terminal, and the vehicle question-answering method includes: S101: The vehicle-mounted terminal receives a user's query statement.
[0026] In an embodiment of the present application, a user can obtain data information of surrounding vehicles by asking the vehicle-mounted terminal. The surrounding vehicles can be the vehicles in front of the vehicle or the vehicles behind the vehicle. An inquiry statement refers to a statement that the user asks the vehicle-mounted terminal, and the inquiry statement includes the position information of the vehicle to be identified. The vehicle to be identified refers to the vehicle that the user wants to inquire about, and the position information refers to the position of the vehicle to be identified. Exemplarily, the inquiry statement may be "What is the vehicle in front of the left?", wherein the left front in the above inquiry statement is the position information, and the vehicle in front of the left is the vehicle to be identified; for another example, the inquiry statement may also be "What is the vehicle directly in front of it?", etc.
[0027] In some examples, at least one receiver, such as a microphone, may be provided in the cabin of the vehicle, and the user may speak the above inquiry statement by voice, and the vehicle-mounted terminal may receive the user's inquiry statement through the above receiver.
[0028] S102: The vehicle-mounted terminal collects images according to the position information of the vehicle to be identified to obtain a target image.
[0029] After receiving the user's query statement, the vehicle-mounted terminal can identify the query statement to obtain the position information of the vehicle to be identified in the query statement, and then perform image acquisition according to the position corresponding to the position information of the vehicle to be identified to obtain the target image.
[0030] S103: The vehicle-mounted terminal detects the target image to obtain the pixel position of the center point of the target vehicle in the target image and the depth information of the target vehicle.
[0031] After the vehicle-mounted terminal obtains the target image, it can perform target detection and depth detection on the target image, which are introduced below respectively.
[0032] The vehicle-mounted terminal can use a target detection algorithm to perform target detection on the target image, thereby obtaining a target frame of the target vehicle in the target image, and then use the center point of the target frame of the target vehicle as the center point of the target vehicle. After determining the center point of the target vehicle, the pixel position of the center point in the target image can be determined.
[0033] It should be noted that the embodiment of the present application does not specifically limit the algorithm used in the target detection process of the target image.
[0034] The vehicle-mounted terminal can use a depth detection algorithm to perform depth detection on the target image, thereby obtaining depth information of each part of the target image, and then can determine the depth information corresponding to the target vehicle according to the pixel position of the target vehicle in the target image. The depth information of the target vehicle can be the depth information corresponding to the center point of the target vehicle; in other examples, the depth information of the target vehicle can be the average of the depth information of multiple positions of the target vehicle.
[0035] S104: The vehicle-mounted terminal determines the direction vector of the target vehicle in the camera coordinate system according to the pixel position and depth information of the target vehicle and the origin position in the camera coordinate system.
[0036] like Figure 2 As shown in the figure, this figure is a schematic diagram of a camera coordinate system provided in an embodiment of the present application. The camera coordinate system includes three directions: x, y and z. The vector represents the direction vector of the i-th target vehicle in the camera coordinate system, where (xi ,y i ) represents the pixel position of the center point of the i-th target vehicle in the target image, z i Represents the depth information of the i-th target vehicle in the target image. The vector Can be based on The three-dimensional coordinates and origin of The three-dimensional coordinates of The three-dimensional coordinates of can be (0,0,0). The above vector That is, the direction vector of the i-th target vehicle in the camera coordinate system.
[0037] S105. The vehicle-mounted terminal calculates the azimuth angle of the target vehicle according to the direction vector and the reference vector.
[0038] The vehicle terminal calculates the direction vector of the i-th target vehicle and the reference vector , calculate the azimuth angle of the i-th target vehicle , the calculation formula is as follows:
[0039] in, represents the azimuth angle of the target vehicle, represents the reference vector, Represents the direction vector of the i-th target vehicle.
[0040] Among them, the reference vector Based on the origin position and reference position Determine, such as Figure 2 , the reference position can be a point The values of the x and y directions of the reference position are 0, and the value of the z direction is a reference value, such as 1, 2, etc.
[0041] It should be noted that, in practical applications, the reference position can be set according to actual conditions or calculation requirements.
[0042] S106: The vehicle-mounted terminal determines whether the azimuth angle of the target vehicle matches the azimuth information of the vehicle to be identified.
[0043] The vehicle-mounted terminal determines whether the azimuth angle of the target vehicle matches the azimuth information of the vehicle to be identified, and obtains a first judgment result. If the first judgment result indicates that the azimuth angle of the target vehicle matches the azimuth information of the vehicle to be identified, S107 is executed. If the first judgment result indicates that the azimuth angle of the target vehicle does not match the azimuth information of the vehicle to be identified, S109 is executed.
[0044] In some embodiments, angle sets corresponding to different azimuth information may be pre-set. For example, the angle set corresponding to the left front is [0,75], the angle set corresponding to the front is [75,105], and the angle set corresponding to the right front is (105,180]. The unit of the angle is degree.
[0045] Taking determining whether the azimuth angle of the i-th target vehicle matches the azimuth information of the vehicle to be identified as an example, the vehicle-mounted terminal can first determine the angle set corresponding to the azimuth information of the vehicle to be identified after calculating the azimuth angle of the i-th target vehicle and the azimuth information in the query statement. If the azimuth angle of the i-th target vehicle is in the angle set corresponding to the azimuth information of the vehicle to be identified, a first judgment result is obtained that the azimuth angle of the i-th target vehicle matches the azimuth information of the vehicle to be identified; if the azimuth angle of the i-th target vehicle is not in the angle set corresponding to the azimuth information of the vehicle to be identified, a first judgment result is obtained that the azimuth angle of the i-th target vehicle does not match the azimuth information of the vehicle to be identified.
[0046] S107: The vehicle terminal identifies the target vehicle and obtains an identification result.
[0047] In some embodiments, after the vehicle-mounted terminal determines that the target vehicle is a vehicle to be identified, the target vehicle can be identified to obtain an identification result. In some application scenarios, the vehicle-mounted terminal can identify information such as the type and brand of the target vehicle.
[0048] In some embodiments, in order to further more accurately identify the target vehicle, after obtaining the depth information of the target vehicle, the vehicle-mounted terminal can use the depth information to compare with the preset depth threshold, specifically, determine the size relationship between the depth information of the target vehicle and the preset depth threshold, and obtain a second judgment result. If the second judgment result indicates that the depth information of the target vehicle is greater than the preset depth threshold, it indicates that the distance between the target vehicle and the vehicle is too far. If the recognition is insisted on, it may lead to incorrect recognition due to factors such as the clarity of the captured image. Therefore, the vehicle-mounted terminal can present a second reply statement to the user, and the second reply statement is used to describe that the vehicle to be identified is not within the recognition range. Among them, the second reply statement can be "The vehicle is not within the recognition range, please try again later." If the second judgment result indicates that the depth information of the target vehicle is less than or equal to the preset depth threshold, it indicates that the distance between the target vehicle and the vehicle is moderate. At this time, the target vehicle is identified, which can increase the accuracy of identifying the target vehicle.
[0049] In other embodiments, the vehicle-mounted terminal may also obtain the pixel area of the target vehicle in the target image, and then determine the size relationship between the pixel area of the target vehicle and the preset area threshold to obtain a third judgment result; if the third judgment result indicates that the pixel area of the target vehicle in the target image is less than the preset area threshold, it indicates that the target vehicle in the target image is too small. If the recognition is insisted on, it may lead to incorrect recognition due to factors such as the small size of the captured image. Therefore, the vehicle-mounted terminal may present a second reply statement to the user, and the second reply statement is used to describe that the vehicle to be recognized is not within the recognition range. If the third judgment result indicates that the pixel area of the target vehicle is greater than or equal to the preset area threshold, it indicates that the size of the target vehicle is moderate. At this time, the target vehicle is recognized, which can increase the accuracy of identifying the target vehicle.
[0050] S108. The vehicle-mounted terminal presents a first reply statement to the user based on the recognition result.
[0051] The recognition result may refer to information such as the brand and type of the target vehicle. After obtaining the recognition result, the vehicle terminal may present a first reply statement to the user. For example, the first reply statement may be "There is a van of brand xx in front of the left side of this vehicle." The vehicle terminal may present the first reply statement to the user through voice.
[0052] If the first judgment result indicates that the azimuth angles of multiple target vehicles match the azimuth information of the vehicle to be identified, the vehicle-mounted terminal identifies each target vehicle, obtains an identification result, and then presents the first reply statement to the user. For example, if there are two target vehicles, the first reply statement may be "The first vehicle in front of the left of this vehicle is a van of brand xx, and the second vehicle is a sedan of brand xx."
[0053] S109. The vehicle-mounted terminal presents a third reply statement to the user.
[0054] In some embodiments, if the azimuth angle of the target vehicle does not match the angle set corresponding to the azimuth information of the vehicle to be identified, a first judgment result is obtained that the azimuth angle of the target vehicle does not match the azimuth information of the vehicle to be identified. In this case, the vehicle-mounted terminal presents a third reply statement to the user, which is used to describe that there is no vehicle in the direction corresponding to the azimuth information of the vehicle to be identified. For example, the third reply statement may be "There is no vehicle in this direction, please try again later."
[0055] Based on the above description, the embodiment of the present application provides a vehicle question-answering method in the field of intelligent cockpit technology, which can be executed by a vehicle-mounted terminal. The method includes: the vehicle-mounted terminal receives the user's query statement, and then performs image acquisition according to the orientation information of the vehicle to be identified in the query statement to obtain a target image, and then detects the target image to obtain the pixel position of the center point of the target vehicle in the target image and the depth information of the target vehicle, and then determines the direction vector of the target vehicle in the camera coordinate system according to the pixel position and depth information of the target vehicle and the origin position in the camera coordinate system, and calculates the orientation angle of the target vehicle based on the direction vector and the reference vector. In this method, the vehicle-mounted terminal does not rely solely on single image information, but also collects depth information in the image, and further generates a three-dimensional vector (i.e., a direction vector) based on the depth information. The orientation angle of the target vehicle determined based on the direction vector introducing the depth information is more accurate, and then when the orientation angle of the target vehicle matches the orientation information of the vehicle to be identified, the target vehicle is identified to obtain an identification result, and finally, according to the identification result, the first reply statement is presented to the user. It can be seen that after the method introduces depth information, the azimuth angle determined by the direction vector generated based on the depth information is more accurate, which can reduce the recognition error and reduce the probability of errors in the vehicle-mounted terminal's answer to the user.
[0056] An embodiment of the present application provides a method for vehicle detection in the field of smart cockpit technology, which includes: data collection, data labeling, manual verification, data processing and model training, etc.
[0057] Data collection refers to collecting vehicle data (including vehicle images), such as RGB images of the vehicle, based on the cockpit scene.
[0058] Data labeling includes two parts. The first part is labeling data related to target detection. First, a small amount of target detection related data is labeled, and then the pre-trained weights of a large detection model with general zero-sample generalization capability (such as grounding dino) are used for fine-tuning. Then, the remaining large batches of data are pre-labeled using the fine-tuned weights. The second part is labeling data related to depth detection. For example, the open source depth estimation model Metric3D v2 can be used to generate pseudo-depth information. This pseudo-depth information can reflect the distance relationship between the vehicle in front and the vehicle. Manual verification refers to the manual verification of the pre-labeling performed using the model.
[0059] Data processing refers to the data processing for target detection tasks, which is to scale the image proportionally, add random noise, Mosiac data enhancement, ColorJitter operation and normalization, and collect a batch of negative sample data, which is sampled in the dataloader with a probability of 0.3. For the data processing of depth estimation tasks, the data enhancement operations used include image flipping, rotation, scaling, translation, color conversion, cropping and filling, adding noise, Gaussian blur, occlusion, affine transformation, geometric perturbation and illumination enhancement, etc., to improve the generalization ability and robustness of the corresponding model.
[0060] Model training means that for target detection tasks, this method mainly includes feature extraction module, multi-scale fusion module, feature enhancement extraction module and task output module. The backbone used by the feature extraction module is Mobilenetv3 small network, and the multi-scale fusion module uses FPN (Feature Pyramid Network). The relevant schematic diagram of the feature enhancement extraction module is as follows Figure 3 As shown in the figure, the task output module includes two output task heads, one corresponding to the Gaussian heat map of the center point of the target box, and the other corresponding to the upper left, lower right, and relative deviation of the target box center.
[0061] like Figure 3 As shown, this figure is a schematic diagram of a feature enhancement extraction module provided in an embodiment of the present application, and the feature enhancement extraction module includes an input module 301, a convolutional block attention module 302, a context module 303 and a merged feature module 304. Among them, the full name of the convolutional block attention module 302 is Convolutional Block Attention Module, abbreviated as CBA.
[0062] The convolutional attention module 302 is composed of a 3×3 convolution kernel with a stride of 1 and padded pixels of 0, a BN (normalization layer, BatchNormalization), and a RELU (rectified linear unit) activation function.
[0063] The context module 303 is composed of an Inconv module, an Upconv module, a Downconv module, and a Downconv2 module. Each of the above modules is composed of a 3*3 convolution kernel with a step size of 1 and a padding pixel of 1, a BN, and a RELU activation function.
[0064] The processing process is as follows: first, use the Inconv module to extract features from the input feature map to obtain a feature map with an output channel number that is one-fourth of the input channel number, then use the Upconv module, Downconv module, and Downconv2 module to extract features respectively, and finally merge the feature maps according to the channel dimension.
[0065] It should be noted that the present application does not specifically limit the methods of target detection and depth detection.
[0066] Combination of the above Figure 1 A vehicle question-and-answer method in the field of smart cockpit technology provided in an embodiment of the present application is introduced in detail. The following will introduce the devices and equipment provided in the embodiment of the present application in conjunction with the accompanying drawings.
[0067] like Figure 4 As shown, this figure is a schematic diagram of a vehicle question-and-answer device in the field of intelligent cockpit technology provided by an embodiment of the present application, and the device includes: an interaction module 201, a collection module 202, a detection module 203, a vector module 204, a calculation module 205, a judgment module 206 and an identification module 207; The interaction module 201 is used to receive a user's query statement, wherein the query statement includes the position information of the vehicle to be identified; The acquisition module 202 is used to acquire an image according to the position information of the vehicle to be identified to obtain a target image; A detection module 203 is used to detect the target image to obtain the pixel position of the center point of the target vehicle in the target image and the depth information of the target vehicle; A vector module 204 is used to determine a direction vector of the target vehicle in the camera coordinate system according to the pixel position and depth information of the target vehicle and the origin position in the camera coordinate system; A calculation module 205, configured to calculate the orientation angle of the target vehicle according to the direction vector and the reference vector, wherein the reference vector is determined based on the origin position and the reference position; The judgment module 206 is used to judge whether the orientation angle of the target vehicle matches the orientation information of the vehicle to be identified, and obtain a first judgment result; The identification module 207 is used to identify the target vehicle and obtain an identification result if the first judgment result indicates that the orientation angle of the target vehicle matches the orientation information of the vehicle to be identified; The interaction module is also used to present a first reply statement to the user based on the recognition result.
[0068] Optionally, the calculation module 205 is specifically used to calculate the azimuth angle of the target vehicle according to the direction vector and the reference vector, including:
[0069] in, represents the azimuth angle of the target vehicle, represents the reference vector, Represents the direction vector of the i-th target vehicle.
[0070] Optionally, the judgment module 206 is specifically used to judge the magnitude relationship between the depth information of the target vehicle and a preset depth threshold to obtain a second judgment result; The step of identifying the target vehicle and obtaining an identification result includes: When the second judgment result indicates that the depth information of the target vehicle is less than or equal to the preset depth threshold, the target vehicle is identified to obtain an identification result.
[0071] Optionally, the judgment module 206 is specifically used to present a second reply statement to the user when the second judgment result indicates that the depth information of the target vehicle is greater than the preset depth threshold, and the second reply statement is used to describe that the vehicle to be identified is not within the identification range.
[0072] Optionally, the judgment module 206 is specifically used to obtain the pixel area of the target vehicle in the target image; Determine the size relationship between the pixel area of the target vehicle and a preset area threshold to obtain a third determination result; The step of identifying the target vehicle and obtaining an identification result includes: When the third judgment result indicates that the pixel area of the target vehicle is greater than or equal to a preset area threshold, the target vehicle is identified to obtain an identification result.
[0073] Optionally, the judgment module 206 is specifically used to judge whether the orientation angle of the target vehicle matches the orientation information of the vehicle to be identified, and obtain a first judgment result, including: If the azimuth angle of the target vehicle is within the angle set corresponding to the azimuth information of the vehicle to be identified, a first judgment result is obtained that the azimuth angle of the target vehicle matches the azimuth information of the vehicle to be identified; If the azimuth angle of the target vehicle does not match the azimuth information of the vehicle to be identified, a first judgment result is obtained that the azimuth angle of the target vehicle does not match the azimuth information of the vehicle to be identified.
[0074] Optionally, the judgment module 206 is specifically used to present a third reply statement to the user if the first judgment result indicates that the orientation angle of the target vehicle does not match the orientation information of the vehicle to be identified, and the third reply statement is used to describe that there is no vehicle at the orientation corresponding to the orientation information of the vehicle to be identified.
[0075] According to the technical field of a smart cockpit of an embodiment of the present application, a vehicle question-and-answer device can correspond to the method described in the embodiment of the present application, and the above-mentioned other operations and / or functions of each module / unit of a vehicle question-and-answer device are respectively to realize Figure 1 For the sake of brevity, the corresponding processes of each method in the illustrated embodiment are not described in detail here.
[0076] The present application also provides a computing device. Figure 5 As shown, the figure is a schematic diagram of a computing device provided in an embodiment of the present application, and the computing device 500 includes a bus 501, a processor 502, a communication interface 503 and a memory 504. The processor 502, the memory 504 and the communication interface 503 communicate with each other through the bus 501.
[0077] The bus 501 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 5 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.
[0078] The processor 502 may be any one or more of a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).
[0079] The communication interface 503 is used for communicating with the outside.
[0080] The memory 504 may include a volatile memory, such as a random access memory (RAM). The memory 504 may also include a non-volatile memory, such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD), or a solid state drive (SSD).
[0081] The memory 504 stores executable codes, and the processor 502 executes the executable codes to perform the aforementioned vehicle question-and-answer method.
[0082] Specifically, in implementing Figure 4 In the case of the embodiment shown, and Figure 4 When each module or unit of a vehicle question-and-answer device described in the embodiment is implemented by software, Figure 4 The software or program code required for the functions of each module / unit in the system may be partially or completely stored in the memory 504. The processor 502 executes the program code corresponding to each unit stored in the memory 504 to perform the aforementioned vehicle question-and-answer method.
[0083] The embodiment of the present application also provides a computer-readable storage medium. The computer-readable storage medium can be any available medium that can be stored by a computing device or a data storage device such as a data center containing one or more available media. The available medium can be a magnetic medium (such as a floppy disk, a hard disk, a tape), an optical medium (such as a DVD), or a semiconductor medium (such as a solid-state hard disk). The computer-readable storage medium includes instructions that instruct the computing device to execute the above-mentioned vehicle question-and-answer method.
[0084] The embodiment of the present application further provides a computer program product, which includes one or more computer instructions. When the computer instructions are loaded and executed on a computing device, the process or function described in the embodiment of the present application is generated in whole or in part.
[0085] The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from one website, computer or data center to another website, computer or data center via wired (e.g., coaxial cable, optical fiber, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) means.
[0086] When the computer program product is executed by a computer, the computer executes any one of the aforementioned vehicle question-and-answer methods. The computer program product may be a software installation package, and when any one of the aforementioned vehicle question-and-answer methods is needed, the computer program product may be downloaded and executed on a computer.
[0087] The descriptions of the processes or structures corresponding to the above-mentioned figures have different emphases. For parts that are not described in detail in a certain process or structure, please refer to the relevant descriptions of other processes or structures.
[0088] The above description is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions within the technical scope disclosed in the present application should be included in the protection scope of the present application.
Claims
1. A vehicle question-answering method in the field of intelligent cockpit technology, characterized in that: The method comprises: receiving a query statement from a user, wherein the query statement includes position information of a vehicle to be identified; According to the position information of the vehicle to be identified, image acquisition is performed to obtain a target image; Detecting the target image to obtain a pixel position of a center point of a target vehicle in the target image and depth information of the target vehicle; Determine the direction vector of the target vehicle in the camera coordinate system according to the pixel position and depth information of the target vehicle and the origin position in the camera coordinate system; Calculating the orientation angle of the target vehicle according to the direction vector and the reference vector, wherein the reference vector is determined based on the origin position and the reference position; Determine whether the orientation angle of the target vehicle matches the orientation information of the vehicle to be identified, and obtain a first determination result; If the first judgment result indicates that the orientation angle of the target vehicle matches the orientation information of the vehicle to be identified, the target vehicle is identified to obtain an identification result; Based on the recognition result, a first reply statement is presented to the user.
2. The method according to claim 1, characterized in that The step of calculating the orientation angle of the target vehicle according to the direction vector and the reference vector comprises: in, represents the azimuth angle of the target vehicle, represents the reference vector, Represents the direction vector of the i-th target vehicle.
3. The method according to claim 1, characterized in that The method further comprises: Determine the magnitude relationship between the depth information of the target vehicle and a preset depth threshold to obtain a second determination result; The step of identifying the target vehicle and obtaining an identification result includes: When the second judgment result indicates that the depth information of the target vehicle is less than or equal to the preset depth threshold, the target vehicle is identified to obtain an identification result.
4. The method according to claim 3, characterized in that The method further comprises: When the second judgment result indicates that the depth information of the target vehicle is greater than the preset depth threshold, a second reply statement is presented to the user, where the second reply statement is used to describe that the vehicle to be identified is not within the identification range.
5. The method according to claim 1, characterized in that The method further comprises: Acquire the pixel area of the target vehicle in the target image; Determine the size relationship between the pixel area of the target vehicle and a preset area threshold to obtain a third determination result; The step of identifying the target vehicle and obtaining an identification result includes: When the third judgment result indicates that the pixel area of the target vehicle is greater than or equal to a preset area threshold, the target vehicle is identified to obtain an identification result.
6. The method according to claim 1, characterized in that The determining whether the orientation angle of the target vehicle matches the orientation information of the vehicle to be identified to obtain a first determination result includes: If the azimuth angle of the target vehicle is within the angle set corresponding to the azimuth information of the vehicle to be identified, a first judgment result is obtained that the azimuth angle of the target vehicle matches the azimuth information of the vehicle to be identified; If the azimuth angle of the target vehicle does not match the azimuth information of the vehicle to be identified, a first judgment result is obtained that the azimuth angle of the target vehicle does not match the azimuth information of the vehicle to be identified.
7. The method according to claim 6, characterized in that The method further comprises: If the first judgment result indicates that the orientation angle of the target vehicle does not match the orientation information of the vehicle to be identified, a third reply statement is presented to the user, wherein the third reply statement is used to describe that there is no vehicle at the orientation corresponding to the orientation information of the vehicle to be identified.
8. A vehicle question-and-answer device in the field of smart cockpit technology, characterized in that: The device comprises: An interactive module, configured to receive a user's query statement, wherein the query statement includes the position information of the vehicle to be identified; An acquisition module, used to acquire images according to the position information of the vehicle to be identified, and obtain a target image; A detection module, used to detect the target image, obtain the pixel position of the center point of the target vehicle in the target image and the depth information of the target vehicle; A vector module, used to determine the direction vector of the target vehicle in the camera coordinate system according to the pixel position and depth information of the target vehicle and the origin position in the camera coordinate system; A calculation module, used for calculating the azimuth angle of the target vehicle according to the direction vector and the reference vector, wherein the reference vector is determined based on the origin position and the reference position; A judgment module, used to judge whether the orientation angle of the target vehicle matches the orientation information of the vehicle to be identified, and obtain a first judgment result; an identification module, configured to identify the target vehicle and obtain an identification result if the first judgment result indicates that the orientation angle of the target vehicle matches the orientation information of the vehicle to be identified; The interaction module is also used to present a first reply statement to the user based on the recognition result.
9. A computing device, characterized in that including memory and processor; One or more computer programs are stored in the memory, and the one or more computer programs include instructions; when the instructions are executed by the processor, the computing device executes the method as claimed in any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium is used to store a computer program, and the computer program is used to execute the method according to any one of claims 1 to 7.